summaryrefslogtreecommitdiff
path: root/kconfiglib.py
AgeCommit message (Collapse)Author
2018-08-23Fix recursive 'source' error reportingUlf Magnusson
The recursive 'source' detection was still fine, but the error reporting had broken due a missed variable renaming. The test suite didn't catch it, because a different type of KconfigError was raised instead, due to a separate error in the test suite (need to include tests/Krecursive{1,2}, since paths are relative to $srctree). Fix the variable name and tighten up the tests to check that the KconfigError message is the one we except. Tighten up the dependency loop detection tests in the same way too.
2018-08-23Fix file descriptor leak for the top-level Kconfig fileUlf Magnusson
Needs to be close()d. The other Kconfig files are close()d in _leave_file(). Could drop the reference somehow too, but an explicit close() is best for PyPy, which doesn't do reference counting.
2018-08-23Flag extra tokens after 'if'/'depends on'/'visible if' expressionsUlf Magnusson
Extra trailing tokens after 'if <expr>', 'depends on <expr>', and 'visible if <expr>' now trigger syntax errors instead of being ignored. Oversight. This indirectly makes Kconfig.eval_expr() detect extra trailing tokens as well.
2018-08-22Add a generic node iteratorUlf Magnusson
Suggested by Mitja Horvat (pinkfluid) in https://github.com/ulfalizer/Kconfiglib/pull/50. Kconfig.node_iter() iterates through all menu nodes in the menu tree in Kconfig order. This saves scripts the trouble of implementing their own tree walking code. Have node_iter() take a 'unique_syms' flag that can be enabled to only include symbols defined in multiple locations once. This is often what you want when generating output (and is used by write_config()). Order is still preserved. Piggyback a fix to a syntax error test comment. Parsing has been tightened up now.
2018-08-22Introduce Kconfig.unique_defined_syms and Kconfig.unique_choicesUlf Magnusson
These are the same as Kconfig.defined_syms and Kconfig.choices, except duplicates are removed. Kconfig order is still preserved. This is almost always what you want when iterating through symbols and choices, as it potentially saves work, avoids generating duplicates when writing output, and still preserves Kconfig order for readability. The old attributes will be kept for backwards compatibility (maybe there's some rare cases where they could be useful too). They're created internally anyway.
2018-08-21Merge Symbol._checked and Symbol._writtenUlf Magnusson
These are never used at the same time, and Symbol._visited is a good name for both. Gets rid of an internal attribute.
2018-08-21Handle multiple definition locations in a nicer wayUlf Magnusson
Instead of precalculating a set() to get unique symbols, precalculate a list with any duplicates from multiple definition locations removed, and preserve the order of the symbols within it. This makes it possible to get rid of the Symbol._written shenanigans in functions that only need to iterate through unique symbols in sorted order, which is all of them except write_config() (because it needs to walk the entire menu tree).
2018-08-21Make auto.conf symbol order match .config symbol orderUlf Magnusson
Not likely that you'd need to inspect it, since it's more of an implementation detail of incremental builds, but it doesn't hurt.
2018-08-21Make header symbol order match .config symbol order againUlf Magnusson
I accidentally broke this when I added the _defined_syms_set optimization. No semantic difference, but having the order match is more readable.
2018-08-21Refactor write_config() a bitUlf Magnusson
Since the 'top_node' menu node itself is skipped, we can start from there and move the tree walk to the beginning of the loop. For an empty configuration (top_node.list set to None) the tree walk immediately discovers that there are no more nodes and returns.
2018-08-18Grammar nitUlf Magnusson
2018-08-18Document that MenuNode.filename is relative to $srctreeUlf Magnusson
..or to the current directory of $srctree isn't set.
2018-08-18Look up the top-level Kconfig file relative to $srctreeUlf Magnusson
Due to an old design braino, the top-level Kconfig filename passed to Kconfig.__init__() wasn't looked up relative to $srctree, breaking out-of-tree usage for e.g. menuconfig. Fixing it required ugliness like srctree = os.environ.get("srctree", "") kconfiglib.Kconfig(os.path.join(srctree, "Kconfig")) Change the behavior of Kconfig.__init__() to look up the top-level Kconfig file relative to $srctree. This means that all Kconfig files (both the top-level file and any source'd files) now use $srctree, which makes the vast majority of scripts just work when running out-of-tree. Also remove the note re. loading a subset of Kconfig files. Saying that the top-level file and all source'd Kconfig files are looked up relative to $srctree should make the behavior clear enough. Print a note about the new behavior whenever the top-level Kconfig file can't be opened, as this change could be breaking for some scripts. This is a slight backwards-compatiblity break, so the major version will be bumped.
2018-08-18Revert "Fix $srctree logic for the top-level Kconfig file"Ulf Magnusson
This reverts commit 8a3999bc708e8468ff79665e3cbdfccd603160e1. I realized that this should go in a major release at least, because it has the potential to break scripts that rely on the old behavior of ignoring $srctree for the filename passed to Kconfig.__init__(). A dummy release will bump the version to 9.4.2. Any future release with the change will be 10.0.0.
2018-08-17rsource documentation nitsUlf Magnusson
Fix some grammar nits, and add a motivation at the end (wanting to create self-contained Kconfig trees that can appear anywhere relative to the top-level Kconfig file).
2018-08-17Fix $srctree logic for the top-level Kconfig fileUlf Magnusson
Due to a major design braino, the top-level Kconfig file passed to Kconfig.__init__() wasn't looked up relative to $srctree, breaking out-of-tree usage for e.g. menuconfig. With this change, Kconfig files are consistently looked up relative to $srctree, which makes a lot more sense. Also remove note re. loading a subset of Kconfig files. Saying that the top-level file and all source'd Kconfig files are looked up relative to $srctree should make the behavior clear enough.
2018-08-14Use += instead of extend()Ulf Magnusson
+= also does an in-place modification for lists, and it's a bit faster. Also get rid of an 'if node.defaults' tests. Both symbols and choices can have defaults, and it's not worthwhile as an optimization either.
2018-08-12Simplify menuconfig-without-prompt warningUlf Magnusson
node.item is already available as 'sym'.
2018-08-12Simplify _warn_select_unsatisfied_deps() a bitUlf Magnusson
select_val was only used in a single place, and there's no real harm in calculating expr_value(self.direct_dep) twice for a warning.
2018-08-11Don't pass encoding= to Popen()Ulf Magnusson
Popen()'s 'encoding' parameter is Python 3.6+ only. Unfortunately, Popen()'s universal_newlines=True without 'encoding' will use the encoding from the environment. Do a manual version instead, so that we can still use the user-specified encoding (usually UTF-8). That might prevent problems on systems that are (poorly) configured to use the C locale.
2018-08-10Improve naming in the custom expr. printing functionsUlf Magnusson
- *_fn() prefixes on functions are a bit silly. It makes more sense for the expr_str() parameter, so keep it there. - Use *_expr_str() instead of just *_str(), to make it clearer that these deal with expressions.
2018-08-10Support custom printing of symbols/choices in expressionsUlf Magnusson
Allow custom output formats for symbols/choices when turning expressions into strings, via a user-supplied callback function (sc_str_fn). This makes things like turning symbols into links in generated documentation and displaying symbol values in the menuconfig interface less hacky to implement. Two new Symbol/Choice.custom_str() functions were added, as passing extra arguments to __str__() is awkward.
2018-07-31Look for '$(' instead of '$' for inline macro expansionsUlf Magnusson
This gives a less confusing KconfigError message for syntax errors like 'config $FOO'.
2018-07-24Restore compatibility with old kernelsUlf Magnusson
Add a small hack to restore compatibility with older (2015-) versions of the Linux kernel. Weird help tokens like -help- and --help--- are now accepted again. Compatibility was originally dropped by commit c19fc11 ("Drop some compatibility and tighten up lexing"), but it turns that people are still using Kconfiglib with older kernels. The new compatibility hack has pretty minimal impact at least.
2018-07-24Use universal newlines mode in $(shell) implementationUlf Magnusson
This prevents e.g. stray \r's in command output on Windows after stripping trailing newlines.
2018-07-20Fix detection of hex literalsUlf Magnusson
Missing 'not'. Add some literals to the Kstrict testcase.
2018-07-20Mention KCONFIG_STRICT in the READMEUlf Magnusson
Also fix up Kconfig.__init__() docstring to say that KCONFIG_STRICT needs to be "y".
2018-07-20Add KCONFIG_STRICT flag for flagging refs. to undefined symsUlf Magnusson
Settings KCONFIG_STRICT to y in the environment turns on warnings for all references to undefined symbols within Kconfig files (with the only gotcha that hex literals must be prefixed by 0x or 0X, to make it possible to distinguish them from undefined references). Always flagging undefined references gets awkward, as some projects (e.g. the Linux kernel) use multiple Kconfig trees with shared files, leading to some safe undefined references. It's helpful for other projects though. Having KCONFIG_STRICT as an environment variable is handy when multiple tools are involved. Piggyback a small README change re. warnings. Kconfiglib now has many more warnings than the C tools.
2018-07-20Simplify self.srctree assignmentUlf Magnusson
2018-07-19Simplify enabling universal newlines modeUlf Magnusson
Get rid of _UNIVERSAL_NEWLINES_MODE ("rU") and just convert "r" into "rU" instead for Python 2. "r" and "w" are the only modes we need. Rename _open_enc() to just _open() as well. It handles universal newlines mode now too. Piggyback a small note on possibly using 'yield' instead in _tokenize().
2018-07-18Add def_int, def_hex, and def_string keywordsUlf Magnusson
Analogous to def_bool and def_tristate, setting the type and adding a default at the same time. This is a Kconfiglib extension. These keywords can be useful in projects that make use of symbols defined in multiple locations, and remove some Kconfig inconsistency.
2018-07-17Factor out isinstance(str) check in set_value()Ulf Magnusson
All types besides bool and tristate require the argument to be a string.
2018-07-17Simplify defconfig_filename() returnsUlf Magnusson
2018-07-17Remove outdated commentUlf Magnusson
'filename' no longer exists, and full_filename is explained in the _enter_file() doc-comment.
2018-07-16Refactor _T_*SOURCE implementation a bitUlf Magnusson
Get rid of the 'relpath(..., srctree) -> join(srctree, ...)', which undid a previous operation. _enter_file() only has a single caller and is more of a helper function, so it's okay if the interface is a bit weird.
2018-07-16Fix outdated _T_*SOURCE commentUlf Magnusson
source statements now always look relative to $srctree.
2018-07-15Formatting nitUlf Magnusson
2018-07-15Switch to more sensible globbing statements (w/ backwards compatibility)Ulf Magnusson
Instead of having 'source' and 'gsource', have 'source' always glob, but require the pattern to match at least one file, throwing KconfigError otherwise. Have separate 'osource' and 'orsource' statements (the o is for "optional") for cases where it's okay for the pattern to not match any files. This is analogous to 'include' and '-include' in Make. The biggest flaw with 'gsource' was that there was no way to do a globbing match while requiring something to match, possibly leading to subtle failures. Preserve backwards compatibility by having "gsource" and "grsource" be aliases for "osource" and "orsource", respectively. Also include some related changes: - Kconfig.srctree is now set to the empty string if $srctree is unset, rather than to None. This gives nice behavior with os.path.join() and os.path.relpath(), which treat the empty string as the current directory (without adding './', for os.path.join()). - When $srctree is set, Kconfig files in the current directory will no longer override Kconfig files in $srctree when the relative paths match. This was likely a bug all along in the C tools, and probably only makes sense for .config files. I've seen it cause breakage in Zephyr. - Clarify the behavior of $srctree in the Kconfig.__init__() docstring. - Make MenuNode.filename be relative to $srctree for the Kconfig file passed to Kconfig.__init__(). This makes it consistent. The major version will be bumped later due to the small Kconfig.srctree API change.
2018-07-15Reorder _filestack fieldsUlf Magnusson
Bit easier to read.
2018-07-15Detect recursive 'source' earlierUlf Magnusson
Off-by-one error, though still functional. The recursive source is now detected immediately as the file is source'd the second time. Also remove an outdated comment re. KconfigError.
2018-07-13Fix absolute $srctree prefixes showing up on gsource'd filesUlf Magnusson
When using gsource with $srctree set to an absolute path, the $srctree prefix would show up in MenuNode.filename, trickling its way into e.g. generated documentation. This was due to a broken test: os.path.isabs() was checked after joining the pattern with $srctree, making it mistake an absolute $srctree for an absolute path in the Kconfig file. Fix the test.
2018-07-11Massively speed up U-Boot parsingUlf Magnusson
U-Boot has a ton of definition locations for some symbols, causing a lot of redundant work when iterating over Kconfig.defined_syms in _build_dep(). Iterate over set(Kconfig.defined_syms) instead, wherever possible. This speeds up the U-Boot parsing time from 4 seconds to 0.6 seconds on my machine. Also update the bundled tools to iterate over set(Kconfig.defined_syms). The performance loss is negligible even for projects that don't use multiple definition locations. Update the documentation to clarify that symbols/choices defined in multiple locations appear multiple times in Kconfig.defined_syms/choices as well.
2018-07-11Comment formatting nitUlf Magnusson
2018-07-10Remove trailing whitespaceUlf Magnusson
2018-07-10Warn if int/hex 'default' is outside active 'range'Ulf Magnusson
Only out-of-range user values generated warnings before. The C tools warn for neither of them.
2018-07-10Warn if int/hex/string symbols are evaluated logicallyUlf Magnusson
They always evaluate to n. Would be nice if there was a trivial way to give the location(s). Just say "somewhere" for now.
2018-07-10Add Kconfig preprocessorUlf Magnusson
Implement the Kconfig preprocessor described in https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.txt (which is now in linux-next and will appear in Linux 4.18). A new Kconfig.variables property holds all the preprocessor variables so that they can be inspected programmatically. Preprocessor variables are represented by a new Variable class. With the preprocessor, environment variables are referenced with $(FOO) instead of $FOO. For backwards compatibility, $FOO is accepted as well for now (and leaves "$FOO" as-is if FOO doesn't exist). The $FOO syntax might be dropped at some point in the future (together with a major version increase). It should be supported for a few months at least. Some internals were cleaned up too, mostly related to parsing. Some outdated documentation was fixed as well.
2018-07-10Generalize select-with-unsatisfied-deps warningUlf Magnusson
y-selecting a symbol with direct dependencies m should be flagged as well. Mirrors a change to the C tools.
2018-07-02Refactor tokenization a bitUlf Magnusson
Have _tokenize() take the string to tokenize and return a list of tokens, and handle all the token list management outside. Simplifies the internal logic a bit. Likely faster too.
2018-07-02Simplify some dict.get() calls by passing defaultsUlf Magnusson