EEP 44: Additional preprocessor directives #

Abstract #

This EEP proposes extensions to the preprocessor to allow more powerful conditional compilation. The existing -ifdef directive provides the bare minimum functionality for doing conditional compilation, but it will often require help from external tools such as autoconf.

Specification #

We will introduce a new predefined macro and four new preprocessor directives.

The OTP_RELEASE macro #

There will be a new predefined macro called OTP_RELEASE. Its value will be an integer giving the release number for run-time system that is running the compiler. In OTP 19, its value will be 19.

Code that must work both in both OTP 18 and OTP 19 can use the following construction:

-ifdef(OTP_RELEASE).
  %% Code that will work in OTP 19 or higher.
-else.
  %% Code that will work in OTP 18.
-endif.

From OTP_RELEASE, information about the minimum capabilities of the run-time system can be inferred. It is especially useful for testing for the presence of major new features, especially language features.

As a hypothetical example, assuming that OTP_RELEASE had been available in OTP 17, if ?OTP_RELEASE == 17 evaluated to true, we would know that maps were supported.

The -if And -elif Directives #

The syntax for the new directives is as follows:

-if(Expression).
   .
   .
   .
-elif(Expression).
   .
   .
   .
-else.
   .
   .
   .
-endif.

The -elif directive may be repeated any number of times.

Expression is similar to the kind of expressions that are allowed in guards, with a few differences:

• Only a single expression is allowed. , and ‘;’ may not be used. Use andalso or orelse instead.

• In addition to the guard BIFs that are allowed in guards, there are several additional functions allowed in the expression for an -if or -elif. Those functions are described in the next section.

• Calls to unknown functions will not cause a compilation error, but an evaluation failure which will cause the lines that follows the -if or -elif to be skipped. See the Examples section for an example to see why that is useful. Calls to BIFs that are not guard BIFs (such as integer_to_list/1) will cause a compilation error.

Built-In Functions in -if/-elif #

The following functions are available in -if and -elif expressions (and only there):

• defined(Symbol)
• is_deprecated(Module, Function, Arity)
• is_exported(Module, Function, Arity)
• is_header(Header)
• is_module(Module)
• version(App)

Descriptions of each if-builtin follow.

defined/1 #

defined(Symbol) tests whether the preprocessor symbol is defined, just like -ifdef(Symbol).

is_deprecated/3 #

is_deprecated(Module, Function, Arity) tests whether Function/Arity is deprecated. It returns true if and only if the compiler would generate a deprecated warning for the function.

To clarify, there are two ways that a function can be deprecated.

• One is by using the -deprecated() attribute. This is what you use to deprecate your functions, and the Xref tool knows about it. The compiler does not, and is_deprecated/3 does not either.

• The other way is by listing the function in the compiler’s table of deprecated functions in the otp_internal module. This is what is_deprecated/3 consults. is_deprecated(M, F, A) is true if and only if M:F/A is listed in that table; the nowarn_deprecated option has no effect on this decision.

is_exported/3 #

is_exported(Module, Function, Arity) tests whether Function/Arity is exported from Module.

Module must already have been compiled. is_exported/3 will first call code:ensure_loaded/1 to load Module if it is not already loaded. If Module is not loaded and code:ensure_loaded/1 fails to load it, is_exported/3 will return false. When Module is known to be loaded, is_exported/3 will test whether the Function/Arity is exported from Module.

is_header/1 #

is_header(Header) tests whether the header file Header exists. It searches for header files in the same way as -include_lib.

is_module/1 #

is_module(Module) tests whether the module Module exists.

Module must already have been compiled. is_module/1 will call code:ensure_loaded/1 to load Module if it is not already loaded. If and only if code:ensure_loaded/1 returns {module,Module}, is_module/1 will return true.

version/1 #

version(App) returns the version number for the given application App as a list of integers and strings.

First the version number string will be split at each “.” to produce a list of strings. Then an attempt will be made to convert each string in the list to an integer using list_to_integer/1. If the conversion fails, the string will be kept.

Here is an example:

"1.10.7"

First the string will be split:

["1","10","7"]

Then each string in the list will be converted to an integer:

[1,10,7]

Here is another example:

"1.6.0c"

First the string will be split:

["1","6","0c"]

Then version/1 will attempt to convert each string to an integer:

[1,6,"0c"]

The last string is not numeric, so it is kept.

The version string is fetched from the app file for the application. If the application cannot be found in the code path, or if the app file cannot be read, or if there is no vsn record in the file, the return value will be [].

The -error Directive #

The syntax for the -error directive is:

-error(Term).

The directive will cause a compilation error. The error message will look like:

file.erl:Line: -error(Term).

Here is an example:

-module(example).
-error("This is wrong").
-error(wrong).
-error("Macros will be expanded: " ?MODULE_STRING).

The error message will be:

example.erl:2: -error("This is wrong").
example.erl:3: -error(wrong).
example.erl:4: -error("Macros will be expanded: example").

The -warning Directive #

The syntax for the -warning directive is:

-warning(Term).

The directive will generate a warning, but the compilation will continue. The warning message will look like:

file.erl:Line: Warning -warning(Term).

Here is an example:

-module(example).
-warning("This module is obsolete").
-warning("Macros will be expanded: " ?MODULE_STRING).

The warning message will be:

example.erl:2: Warning: -warning("This module is obsolete").
example.erl:3: Warning: -warning("Macros will be expanded: example").

Examples #

Here is an example of code that will work in OTP 18 through OTP 20. There will be a compilation error if an attempt is made to compile the code in OTP 21 or higher.

-ifndef(OTP_RELEASE).
  %% Code that will work in OTP 18.
-else.
  %% OTP 19 or higher.
  -if(?OTP_RELEASE =:= 19).
    %% Code that will work in OTP 19.
  -elif(?OTP_RELEASE =:= 20).
    %% Code that will work in OTP 20.
  -else.
    -error("Unsupported OTP release").
  -endif.
-endif.

(Note that current versions of the preprocessor has partial support for -if in that it can skip an -if … -endif construction. Therefore this code example will work in OTP 18.)

Here is an hypothetical example showing how a problem could have been solved in the past (see predefined Erlang version macros).

-if(is_module(ssh_daemon_channel)).
  %% R16B: use new ssh behaviour
  -behavior(ssh_daemon_channel).
-else.
  %% R15: use old ssh behaviour
  -behaviour(ssh_channel).
-endif.

Here is an example of dealing with a newly introduced header file.

-if(is_header("stdlib/include/assert.hrl")).
  -include_lib("stdlib/include/assert.hrl").
-else.
  %% Define dummy macros just so that our code will compile.
  -define(assert(E),ok).
  -define(assertNot(E),ok).
-endif.

Here is an hypothetical example showing how we could have tested for the presence of maps:

-if(not is_map(a)).
  %% The guard BIF is_map/1 exists, i.e. maps are supported.
-else.
  %% No support for maps in this release.
-endif.

Note that not is_map(a) will evaluate to true if the is_map/1 is a supported guard BIF. If is_map/1 is not a supported guard BIF, the call to is_map/1 will generate an exception which will fail the expression.

Here is an example involving the hypothetical foobar application. Since it is not included in OTP, it might not have been compiled, and is_exported/3 could return false for the wrong reason. To guard against that, we will abort the compilation if the foobar module does not exist:

-if(not is_module(foobar)).
-error("The foobar application has not been compiled").
-endif.

-if(is_exported(foobar, new_feature, 1)).
%% Do something smart with the new feature.
-else.
%% Do as best as we can without the new feature.
-endif.

Motivation #

It is common practice for many open-source applications (or libraries) to work with at least two major releases of OTP: the current release and the previous one. An application may also have dependencies to other third-party libraries and may need to work with different versions of those.

Some applications may support several releases by refraining from using features that are not available in both releases. That may not always be possible, depending on the purpose of the application. A tool for pretty printing Erlang terms, for example, would not be very useful if it didn’t support all data types in the release in which it was running.

There is also another issue. Modern applications are expected:

• To compile without any warnings. Many developers use -Werror to turn warnings into compilation errors. That means that warnings for deprecated functions must be suppressed or eliminated. As an example, the now/0 BIF was marked as deprecated in OTP 18. The recommended replacement BIFs were introduced in the same release.

• Not to cause any warnings in Dialyzer, and to have good type specifications for all exported functions to help finding errors. The type specifications must compile in all supported releases, and must not cause warnings.

In many cases, the most practical solution for supporting several OTP releases is conditional compilation, that is, if some condition if fulfilled, one part of a source file will be compiled, and another part if not. For example, to handle the deprecation of now/0:

-ifdef(NOW_DEPRECATED).
  %% Use the recommended replacement functions.
  sys_time() ->
    erlang:timestamp().
  uniq_name() ->
    Uniq = erlang:unique_integer([positive]),
    lists:flatten(io_lib:format("name_~w", [Uniq]));
-else.
  %% Use now/0.
  sys_time() ->
    now().
  uniq_name() ->
    {A,B,C} = now(),
    lists:flatten(io_lib:format("name_~w_~w_~w", [A,B,C])).
-endif.

That approach works, but some external tool (for example autoconf) will have to arrange for -DNOW_DEPRECATED to be added to the command line for erlc if now/0 has been deprecated.

Our suggestion for extending the preprocessor facilitates using conditional compilation without any external tools. Assuming that the extended preprocessor had been available earlier, the previous example can be rewritten to:

-if(is_exported(erlang, timestamp, 0)).
  %% Use the recommended replacement functions.
  sys_time() ->
    erlang:timestamp().
  uniq_name() ->
    Uniq = erlang:unique_integer([positive]),
    lists:flatten(io:lib_format("name_~w", [Uniq])).
-else.
  %% Use now/0.
  sys_time() ->
    now().
  uniq_name() ->
    {A,B,C} = now(),
    lists:flatten(io:lib_format("name_~w_~w_~w", [A,B,C])).
-endif.

Alternatively, the first -if could have been written:

-if(is_deprecated(erlang, now, 0)).

Rationale #

Preprocessors have a bad reputation, so why extend the preprocessor?

A quick Google search for “preprocessor evil” seems to indicate that it is the macro expansion in the preprocessor that is considered evil, not the conditional compilation part.

That said, the major pitfall of conditional compilation is that the code may be misbehave if it is run in a different environment than it was compiled in. This potential problem already exists with the -ifdef directive in the current preprocessor. It is the responsibility of the user of conditional compilation to ensure that the code is run in an environment compatible with the compilation environment.

There is one thing a preprocessor, and only a preprocessor, can do: skip code that is not syntactically correct (for example, code that uses the map syntax). Therefore, it seems that there is no way getting around using a preprocessor. We could invent a new preprocessor, but that is not the purpose of this EEP.

What about feature detection instead of testing version numbers?

We are all for that. Whenever possible, tests against version numbers should be avoided if there is a better way. For example, to test whether the new behavior ssh_daemon_channel exists, use is_module(ssh_daemon_channel).

Would it not be better to have a built-in supported function to test for language-related features instead of testing for the OTP release number?

-if(supported(maps)).
%% Map code.
-endif.

Perhaps. It seems that for this to work the list of supported feature names accepted by supported must be carefully maintained and documented for each release. The users must then look up the appropriate feature name to use. It may also not be obvious how to name minor changes to the type specification syntax or to the language itself. The resulting code may not be any easier to understand than a test against a release number.

Rationale for allowing unknown functions in expressions #

The rules for expressions says that the following example is legal because foobar/0 is an unknown function:

-if(foobar()).
%% Always skipped.
-endif.

The reason is that otherwise some expressions would be legal in some releases but not others. For example, -if(is_map(a)) would be legal in OTP releases that support maps, but cause a compilation error in other releases. Also, as a side effect, testing guard BIFs can be used to test for new features instead of testing OTP_RELEASE.

Backwards Compatibility #

Modules that define the OTP_RELEASE macro will fail to compile with a message similar to this:

example.erl:4: redefining predefined macro 'OTP_RELEASE'

Similarly, attempting to define OTP_RELEASE from the command line using -D will also fail.

Modules that has attributes that looks like -error(Term) or -warning(Term) will need to be updated, as -error(Term) will now cause a compilation error and -warning(Term) will cause a compilation warning.

The function epp:parse_erl_form/1 can now return {warning,Info} in addition to its previous return values. Applications that call epp:parse_erl_form/1 will need to be updated to handle the new return value. Similarly, the epp:parse_file() family of functions can now include {warning,Info} tuples in the returned list of forms.

Implementation #

The reference implementation can be fetched from Github like this:

git fetch git://github.com/bjorng/otp.git bjorn/preprocessor-extensions

Copyright #

This document has been placed in the public domain.