Professional CMake:

A Practical Guide

Go beyond trivial examples and learn how to use CMake effectively with practical advice direct from a CMake co-maintainer

 

Do Not Redefine CMake Commands

In this article, we take a closer look at a particular example from the popular Effective CMake talk by Dan Pfeifer. The example in question relies on undocumented behavior, the dangers of which may not be immediately obvious and can lead to infinite recursion.

In the section of the talk where package management is discussed, the following example code is presented (around 52m38s):

set(CMAKE_PREFIX_PATH "/prefix")
set(as_subproject Foo)

macro(find_package)
  if(NOT "${ARG0}" IN_LIST as_subproject)
    _find_package(${ARGV})
  endif()
endmacro()

add_subdirectory(Foo)
add_subdirectory(App)

The intent behind the above block of CMake code is to redefine the built-in find_package() command such that it checks if the package to be found is one of those named in the as_subproject list variable. If it is in that list, then it should be assumed that the package will be added to the project directly via add_subdirectory() and the find_package() call should do nothing. Otherwise, the call should be forwarded to the original built-in find_package() command. It relies on undocumented behavior to call through to the original implementation using _find_package().

(The following explanation is largely extracted from the Functions And Macros chapter of the book Professional CMake: A Practical Guide)

When function() or macro() is called to define a new command, if a command already exists with that name, the undocumented CMake behavior is to make the old command available using the same name except with an underscore prepended. This applies whether the old name is for a built-in command, a custom function or a macro. If a command is only ever overridden once, techniques like in the example above appear to work, but if the command is overridden again, then the original command is no longer accessible. The prepending of one underscore to “save” the previous command only applies to the current name, it is not applied recursively to all previous overrides. This has the potential to lead to infinite recursion, as the following contrived example demonstrates:

function(printme)
    message("Hello from first")
endfunction()

function(printme)
    message("Hello from second")
    _printme()
endfunction()

function(printme)
    message("Hello from third")
    _printme()
endfunction()

printme()

One may naively expect the output to be as follows:

Hello from third
Hello from second
Hello from first

But instead, the first implementation is never called because the second one ends up calling itself in an infinite loop. When CMake processes the above, here is what occurs:

  1. The first implementation of printme() is created and made available as a command of that name. No command by that name previously existed, so no further action is required.
  2. The second implementation of printme() is encountered. CMake finds an existing command by that name, so it defines the name _printme to point to the old command and sets printme to point to the new definition.
  3. The third implementation of printme() is encountered. Again, CMake finds an existing command by that name, so it redefines the name _printme to point to the old command (which is the second implementation) and sets printme to point to the new definition.

When printme() is called, execution enters the third implementation, which calls _printme(). This enters the second implementation which also calls _printme(), but _printme() points back at the second implementation again and infinite recursion results. Execution never reaches the first implementation.

For the find_package() example mentioned earlier, the implications of find_package() being redefined more than once are catastrophic.

  • The original find_package() command becomes permanently inaccessible.
  • Any attempt to call find_package() will result in infinite recursion if _find_package() is called by the new implementation.

In all fairness, in his talk Dan highlighted that he envisioned that the code sample in question would only ever be executed once as part of a package manager. The find_package() macro redefinition would be inserted by a package manager as part of how it incorporated external dependencies under its control. Even if find_package() were only redefined once though, it would still be relying on undocumented CMake behavior which may be modified or removed completely in a future version. Reliance on such behavior should be discouraged and as the above discussion shows, the technique is not safe to use in general.

In a future article, we will also look at another aspect of this example to discuss some potential traps when forwarding command arguments. Those gotchas will add further weight to the recommendation to avoid overriding existing commands.


Get the book for more CMake content

Book cover, Professional CMake: A Practical Guide

3 thoughts on “Do Not Redefine CMake Commands”

  1. What sucks is that there are only a handful of talks like this, and they all approach the problem slightly differently. Meanwhile, as a new user of cmake, building an open source project, I almost immediately ran into this (rather severe) weakness of cmake and have been looking for how to resolve it, but keep getting confused by the slight differences (hacks) in approach by these talks, as well as a an almost total black out on the Internet.

    It seems to me, in conclusion, that these are FUNDAMENTAL and unsolved problems.

    I would like 2 things to be added to CMAKE officially, so that we are not hacking around and creating trouble just to do the basics:

    1) An add_subdirectory command that allows for a few extract options, include the ability to do the same stuff as ExternalProject_Add, except that it
    a) works at CONFIGURE time to download the directory from GIT etc.
    b) takes an extra parameter to prefix all targets in the subdirectory with a namespace, preventing namespace pollution

    one possible way to do this is to change/copy/modify ExternalProject_Add so that it supports use during configure time via an option, and then use a standardized naming system such that the system can tie find_package together with ExternalProject_Add using the name given as the first parameter given to it. In other words, I need a way to tie ExternalProject together with find_package so they work together.

    2) Upgrade find_package to support subdirectory adding and working with target names instead of old fashion file names to be searched on the filesystem. Add standardized variable names to control turning on subdirectory mode as well as specifying the subdir to use:
    _USE_AS_SUBDIR
    _SUBDIR

    or something like that. The user would set these, then use the normal find_package() command which would

    • An add_subdirectory command that allows for a few extract options, include the ability to do the same stuff as ExternalProject_Add, except that it
      a) works at CONFIGURE time to download the directory from GIT etc.

      See the FetchContent module

      b) takes an extra parameter to prefix all targets in the subdirectory with a namespace, preventing namespace pollution

      The lack of a namespace feature for targets is something I too have wanted since working on FetchContent. I’m not aware of anyone working on such a capability at the moment though.

      In other words, I need a way to tie ExternalProject together with find_package so they work together.

      In CMake 3.14, I added the FetchContent_MakeAvailable() function to the FetchContent module. I named it that way specifically because I’m trying to work it towards supporting both add_subdirectory() and find_package() as ways to bring a dependency into the build. It’s not trivial though, see the (long) discussions in this issue for some of the details.

    • I would very much like that all the C-Make Tutors work together on a set of guidelines, with clear cut how-to’s.

      That being said, I really dont think you should use CMake for building external dependencies. You either have to do that when configuring or you lack the outputs for find_package. Both are bad solutions, and this is an limitation CMake will always have.

      It gets worse if you use a project that does that, you might not want the version the project has coded in its Makefiles.
      It gets alot worse if you want the option to be able to use any of the following variants by choice: system installed, staged (prepared output in another directory) or built from source.

      I found another tool layer to be better fitting, and just using find_package as the binaries will be somehow available by the time CMake configures your project. Conan fits this purpose pretty well and seems the way forward.

Leave a Reply (all comments are moderated)

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: