From 12289b080b33f6dc8fb73936a8c5d0ffa1cb63fa Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 07:32:42 +0100 Subject: Update intro in README.rst Copied straight from the module docstring, with some formatting added. Need to update other parts too. --- README.rst | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 82 insertions(+), 11 deletions(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index 31c3a00..19bf535 100644 --- a/README.rst +++ b/README.rst @@ -1,17 +1,88 @@ -Kconfiglib -========== +.. contents:: Table of contents + :backlinks: none -A Python library for doing stuff with `Kconfig `_-based -configuration systems. Can extract information, query and set symbol values, -and read and write ``.config`` files. Highly compatible with the -``scripts/kconfig/*conf`` utilities in the kernel, usually invoked via make -targets such as ``menuconfig`` and ``defconfig``. +Overview +-------- -Supports both Python 2 and Python 3 without modification, and should also run -on non-\*nix platforms +Kconfiglib is a Python 2/3 library for scripting and extracting information +from `Kconfig +`_ +configuration systems. It can do the following, among other things: + +- **Programmatically get and set symbol values** + + `allnoconfig.py `_ and `allyesconfig.py + `_ examples are provided, automatically verified to + produce identical output to the standard ``make allnoconfig`` and ``make + allyesconfig``. + +- **Read and write .config files** + + The generated ``.config`` files are character-for-character identical to what + the C implementation would generate (except for the header comment). The test + suite relies on this, as it compares the generated files. + +- **Inspect symbols** + + Printing a symbol gives output which could be fed back into a Kconfig parser + to redefine it***. The printing function (``__str__()``) is implemented with + public APIs, meaning you can fetch just whatever information you need as + well. + + A helpful ``__repr__()`` is implemented on all objects too, also implemented + with public APIs. + + ***Choice symbols get their parent choice as a dependency, which shows up as + e.g. ``prompt "choice symbol" if `` when printing the symbol. This + could easily be worked around if 100% reparsable output is needed. + +- **Inspect expressions** + + Expressions use a simple tuple-based format that can be processed manually + if needed. Expression printing and evaluation functions are provided, + implemented with public APIs. + +- **Inspect the menu tree** + + The underlying menu tree is exposed, including submenus created implicitly + from symbols depending on preceding symbols. This can be used e.g. to + implement menuconfig-like functionality. + + See the `menuconfig.py `_ example. + + +Here are some other features: + +- Single-file implementation. The entire library is contained in `kconfiglib.py + `_. + +- Runs unmodified under both Python 2 and 3. The code mostly uses basic Python + features and has no third-party dependencies. The most advanced things used + are probably ``@property`` and ``__slots__``. + +- Robust and highly compatible with the standard Kconfig C tools: The test + suite automatically compares output from Kconfiglib and the C tools by + diffing the generated ``.config`` files for the real kernel Kconfig and + defconfig files, for all ARCHes. + + This currently involves comparing the output for 36 ARCHes and 498 defconfig + files (or over 18000 ARCH/defconfig combinations in "obsessive" test suite + mode). All tests are expected to pass. + + A suite of selftests is included as well. + +- Reasonably optimized and not horribly slow despite being a pure Python + implementation: The `allyesconfig.py `_ example + currently runs in about 1.6 seconds on a Core i7 2600K (with a warm file + cache), where half a second is overhead from ``make scriptconfig`` (see below). + + For long-running jobs, `PyPy `_ gives a big performance + boost. CPython is faster for short-running jobs as PyPy needs some time to + warm up. + +- Internals that (mostly) mirror the C implementation while being simpler to + understand. -.. contents:: Table of contents - :backlinks: none Installation ------------ -- cgit v1.2.3 From 4a55a4e0a406a06726673c3eef9e0cb523bf3ce6 Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 07:47:23 +0100 Subject: Clean up intro a bit --- README.rst | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index 19bf535..4249347 100644 --- a/README.rst +++ b/README.rst @@ -32,7 +32,7 @@ configuration systems. It can do the following, among other things: A helpful ``__repr__()`` is implemented on all objects too, also implemented with public APIs. - ***Choice symbols get their parent choice as a dependency, which shows up as + \***Choice symbols get their parent choice as a dependency, which shows up as e.g. ``prompt "choice symbol" if `` when printing the symbol. This could easily be worked around if 100% reparsable output is needed. @@ -53,35 +53,41 @@ configuration systems. It can do the following, among other things: Here are some other features: -- Single-file implementation. The entire library is contained in `kconfiglib.py - `_. +- **Single-file implementation** + + The entire library is contained in `kconfiglib.py `_. -- Runs unmodified under both Python 2 and 3. The code mostly uses basic Python - features and has no third-party dependencies. The most advanced things used - are probably ``@property`` and ``__slots__``. +- **Runs unmodified under both Python 2 and Python 3** + + The code mostly uses basic Python features and has no third-party + dependencies. The most advanced things used are probably ``@property`` and + ``__slots__``. -- Robust and highly compatible with the standard Kconfig C tools: The test - suite automatically compares output from Kconfiglib and the C tools by - diffing the generated ``.config`` files for the real kernel Kconfig and +- **Robust and highly compatible with the standard Kconfig C tools** + + The test suite automatically compares output from Kconfiglib and the C tools + by diffing the generated ``.config`` files for the real kernel Kconfig and defconfig files, for all ARCHes. This currently involves comparing the output for 36 ARCHes and 498 defconfig files (or over 18000 ARCH/defconfig combinations in "obsessive" test suite mode). All tests are expected to pass. - A suite of selftests is included as well. + A comprehensive suite of selftests is included as well. -- Reasonably optimized and not horribly slow despite being a pure Python - implementation: The `allyesconfig.py `_ example - currently runs in about 1.6 seconds on a Core i7 2600K (with a warm file - cache), where half a second is overhead from ``make scriptconfig`` (see below). +- **Not horribly slow despite being a pure Python implementation** + + The `allyesconfig.py `_ example currently runs in + about 1.6 seconds on a Core i7 2600K (with a warm file cache), where half a + second is overhead from ``make scriptconfig`` (see below). For long-running jobs, `PyPy `_ gives a big performance boost. CPython is faster for short-running jobs as PyPy needs some time to warm up. -- Internals that (mostly) mirror the C implementation while being simpler to - understand. +- **Internals that (mostly) mirror the C implementation** + + While being simpler to understand. Installation -- cgit v1.2.3 From 54105994432eb905269d71e9e52dd408604aeb80 Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 08:05:17 +0100 Subject: Update the documentation section Also move it directly after the intro. --- README.rst | 48 ++++++++++++++++++++++++++---------------------- 1 file changed, 26 insertions(+), 22 deletions(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index 4249347..cff59cb 100644 --- a/README.rst +++ b/README.rst @@ -88,6 +88,32 @@ Here are some other features: - **Internals that (mostly) mirror the C implementation** While being simpler to understand. + +Documentation +------------- + +Kconfiglib comes with extensive documentation in the form of docstrings. To view it, run e.g. +the following command: + +.. code:: sh + + $ pydoc kconfiglib + +For HTML output, add ``-w``: + +.. code:: sh + + $ pydoc -w kconfiglib + +A good starting point is to read the module docstring (which you could also just read directly +at the beginning of `kconfiglib.py `_). It gives an introduction to symbol +values, the menu tree, and expressions. + +After reading the module docstring, a good next step is to read the ``Kconfig`` class +documentation, and then the documentation for the ``Symbol``, ``Choice``, and ``MenuNode`` +classes. + +Please tell me if something is unclear to you or can be explained better. Installation @@ -144,28 +170,6 @@ The entire library is contained in `kconfiglib.py `_. Just drop it somewhere. -Documentation -------------- - -The (extensive) documentation is generated by running the following command in the same -directory as `kconfiglib.py `_: - -.. code:: sh - - $ pydoc kconfiglib - -For HTML output, use - -.. code:: sh - - $ pydoc -w kconfiglib - -You could also browse the docstrings directly in -`kconfiglib.py `_. - -Please tell me if something is unclear to you or can be explained better. The Kconfig -language has some dark corners. - Examples -------- -- cgit v1.2.3 From 8c3ef8bbb4f4b3164740e013821f333abd4682af Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 08:38:05 +0100 Subject: Reference module docstring for Linux installation Less documentation duplication. --- README.rst | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index cff59cb..bb4c596 100644 --- a/README.rst +++ b/README.rst @@ -135,33 +135,7 @@ All releases have a corresponding tag in the git repository, e.g. ``v1.0.6``. Installation for the Linux kernel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -After installing with ``pip(3)``, apply ``makefile.patch`` -by running e.g. the following commands in the kernel root: - -.. code:: sh - -   $ wget https://raw.githubusercontent.com/ulfalizer/Kconfiglib/master/makefile.patch - $ git am makefile.patch - -If you do not wish to install anything, the following manual approach will work as well: - -.. code:: sh - - $ git clone git://github.com/ulfalizer/Kconfiglib.git - $ git am Kconfiglib/makefile.patch - -(Warning: The directory name ``Kconfiglib/`` is significant in this case, because it's added to ``PYTHONPATH`` by the new targets in ``makefile.patch``.) - -In addition to creating a handy interface, the make targets created by the -patch (``scriptconfig`` and ``iscriptconfig``) are needed to pick up environment -variables set in the kernel makefiles and later referenced in the Kconfig files -(``ARCH``, ``SRCARCH``, and ``KERNELVERSION`` as of Linux v4.14.0-rc1). -The documentation explains how the make targets are used. The compatibility -tests in the test suite also needs them. - -Please tell me if the patch does not apply. It should be trivial to apply -manually, as it's just a block of text that needs to be inserted near the other -``*conf:`` targets. +See the module docstring at the top of `kconfiglib.py `_. Manual installation ~~~~~~~~~~~~~~~~~~~ -- cgit v1.2.3 From 966567ae9646c1cafe045c97e5a2613ecf7f973b Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 08:45:59 +0100 Subject: Remove outdated notes Kconfiglib 2 has proper menuconfig support, and currently uses __slots__ too. --- README.rst | 53 +++++------------------------------------------------ 1 file changed, 5 insertions(+), 48 deletions(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index bb4c596..d4a5f4a 100644 --- a/README.rst +++ b/README.rst @@ -194,54 +194,11 @@ to the test suite would make sense. Notes ----- -* **Useful information can be extracted from internal data structures.** The - expression format is pretty simple for example: ``A && B && (!C || D == 3)`` is - represented as the tuple structure - ``(_AND, A, (_AND, B, (_OR, (_NOT, C), (_EQUAL, D, 3))))``; see the - ``Config._parse_expr()`` docstring. - - It's hard to come up with good APIs for dealing with expressions given how - general they are, so feel free to look at them directly if none of the - exposed APIs will suffice (modifying them is dangerous though, because it - breaks dependency tracking). Maybe I'll officially document the expression - format and add a bunch of accessors later. The internal format is unlikely - to change in either case, and would probably be returned directly. - - If you come up with some good generally-usable APIs involving - expressions, please tell me. Make sure they also make sense for expressions - involving ``||`` (or) and ``!`` (not). - -* Kconfiglib works well with `PyPy `_. It gives a nice speedup - over CPython when batch processing a large number of configurations (like - the test suite does). - -* Kconfiglib assumes the modules symbol is ``MODULES`` and will warn if - ``option modules`` is set on some other symbol. Let me know if this is a - problem for you. Adding proper ``option modules`` support should be pretty - easy. - -* At least two things make it awkward to replicate a ``menuconfig``-like - interface in Kconfiglib at the moment (but see - `SConf `_, as mentioned above). - - * There are no good APIs for figuring out what other symbols change in value - when the value of some symbol is changed, to allow for "live" updates - in the configuration interface. The simplest workaround is to refetch the - value of each currently visible symbol every time a symbol value is - changed. - - * ``menuconfig`` sometimes creates cosmetic menus implicitly by looking at - dependencies. For example, a list of symbols where all symbols depend on - the first symbol creates a cosmetic menu rooted at the first symbol. - Recreating such menus is awkward. - - There is already basic support internally though, because it's needed to -   get obscure ``choice`` behavior right. See ``_determine_actual_symbols()`` and - its helper ``_has_auto_menu_dep_on()``. - -* Using `__slots__ `_ - on classes would speed things up a bit and save memory. It'd remove some - flexibility though. +* Kconfiglib assumes the modules symbol is ``MODULES``, which is backwards-compatible. + A warning is printed by default if ``option modules`` is set on some other symbol. + + Let me know if you need proper ``option modules`` support. It wouldn't be that + hard to add. * `fpemud `_ has put together `Python bindings `_ to internal functions in the C -- cgit v1.2.3 From 5a1ce75d7795032b112c25cb9c35c3d7edb2bc10 Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 11:34:42 +0100 Subject: Add a sample 'make scriptconfig' To demonstrate some different possibilities. Will need to move some stuff around later too. --- README.rst | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) (limited to 'README.rst') diff --git a/README.rst b/README.rst index d4a5f4a..c6d138a 100644 --- a/README.rst +++ b/README.rst @@ -147,6 +147,91 @@ Just drop it somewhere. Examples -------- +Sample ``make iscriptconfig`` session +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following log should give some idea of the functionality available in the API: + +.. code-block:: + + $ make iscriptconfig + A Kconfig instance 'kconf' for the architecture x86 has been created. + >>> kconf # Calls Kconfig.__repr__() + + >>> kconf.mainmenu_text # Expanded main menu text + 'Linux/x86 4.14.0-rc7 Kernel Configuration' + >>> kconf.top_node # The implicit top-level menu + + >>> kconf.top_node.list # First child menu node + + >>> print(kconf.top_node.list) # Calls MenuNode.__str__() + config SRCARCH + string + option env="SRCARCH" + default "x86" + + >>> sym = kconf.top_node.list.next.item # Item contained in next menu node + >>> print(sym) # Calls Symbol.__str__() + config 64BIT + bool + prompt "64-bit kernel" if ARCH = "x86" + default ARCH != "i386" + help + Say yes to build a 64-bit kernel - formerly known as x86_64 + Say no to build a 32-bit kernel - formerly known as i386 + + >>> sym # Calls Symbol.__repr__() + + >>> sym.assignable # Currently assignable values (0, 1, 2 = n, m, y) + (0, 2) + >>> sym.set_value(0) # Set it to n + True + >>> sym.tri_value # Check the new value + 0 + >>> sym = kconf.syms["X86_MPPARSE"] # Look up symbol by name + >>> print(sym) + config X86_MPPARSE + bool + prompt "Enable MPS table" if (ACPI || SFI) && X86_LOCAL_APIC + default "y" if X86_LOCAL_APIC + help + For old smp systems that do not have proper acpi support. Newer systems + (esp with 64bit cpus) with acpi support, MADT and DSDT will override it + + >>> default = sym.defaults[0] # Fetch its first default + >>> sym = default[1] # Fetch the default's condition (just a Symbol here) + >>> print(sym) # Print it. Dependencies are propagated to properties, like in the C implementation. + config X86_LOCAL_APIC + bool + default "y" if X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI + select IRQ_DOMAIN_HIERARCHY if X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI + select PCI_MSI_IRQ_DOMAIN if PCI_MSI && (X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI) + + >>> sym.nodes # Show the MenuNode(s) associated with it + [] + >>> kconfiglib.expr_str(sym.defaults[0][1]) # Print the default's condition + 'X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI' + >>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it (0 = n) + 0 + >>> kconf.syms["64BIT"].set_value(2) + True + >>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it again (2 = y) + 2 + >>> kconf.write_config("myconfig") # Save a .config + >>> ^D + $ cat myconfig + # Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib) + CONFIG_64BIT=y + CONFIG_X86_64=y + CONFIG_X86=y + CONFIG_INSTRUCTION_DECODER=y + CONFIG_OUTPUT_FORMAT="elf64-x86-64" + CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig" + CONFIG_LOCKDEP_SUPPORT=y + CONFIG_STACKTRACE_SUPPORT=y + CONFIG_MMU=y + ... + * The `examples/ `_ directory contains simple example scripts. See the documentation for how to run them. * `genboardscfg.py `_ from `Das U-Boot `_ generates some sort of legacy board database by pulling information from a newly added Kconfig-based configuration system (as far as I understand it :). -- cgit v1.2.3 From 0a106fcea131fcf88336ef279f76e785dc041dcb Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 12:08:47 +0100 Subject: Add a separate example script README section --- README.rst | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) (limited to 'README.rst') diff --git a/README.rst b/README.rst index c6d138a..237df8c 100644 --- a/README.rst +++ b/README.rst @@ -147,6 +147,61 @@ Just drop it somewhere. Examples -------- +Example scripts +~~~~~~~~~~~~~~~ + +The ``examples/`` subdirectory contains some simple example. Among these are the following ones: + +- `allnoconfig.py `_, `allnoconfig_simpler.py `_, and `allyesconfig.py `_ implement ``make allnoconfig`` and ``make allyesconfig`` in various ways. Demonstrates menu tree walking and value setting. + +- `defconfig.py `_ has the same effect as going into ``make menuconfig`` and immediately saving and exiting. + +- `eval_expr.py `_ evaluates an expression in the context of a configuration. + +- `find_symbol.py `_ searches through expressions to find references to a symbol, also printing a "backtrace" with parents for each reference found. + +- `help_grep.py `_ searches for a string in all help texts. + +- `print_tree.py `_ prints a tree of all configuration items. + +- `menuconfig.py `_ implements a configuration interface that uses notation similar to ``make menuconfig``. It's deliberately kept as simple as possible to demonstrate just the core concepts, and isn't something you'd actually want to use. Here's a screenshot: + +.. code-block:: + + ======== Example Kconfig configuration ======== + + [*] Enable loadable module support (MODULES) + Bool and tristate symbols + [*] Bool symbol (BOOL) + [ ] Dependent bool symbol (BOOL_DEP) + < > Dependent tristate symbol (TRI_DEP) + [ ] First prompt (TWO_MENU_NODES) + < > Tristate symbol (TRI) + [ ] Second prompt (TWO_MENU_NODES) + *** These are selected by TRI_DEP *** + < > Tristate selected by TRI_DEP (SELECTED_BY_TRI_DEP) + < > Tristate implied by TRI_DEP (IMPLIED_BY_TRI_DEP) + String, int, and hex symbols + (foo) String symbol (STRING) + (747) Int symbol (INT) + (0xABC) Hex symbol (HEX) + Various choices + -*- Bool choice (BOOL_CHOICE) + --> Bool choice sym 1 (BOOL_CHOICE_SYM_1) + Bool choice sym 2 (BOOL_CHOICE_SYM_2) + {M} Tristate choice (TRI_CHOICE) + < > Tristate choice sym 1 (TRI_CHOICE_SYM_1) + < > Tristate choice sym 2 (TRI_CHOICE_SYM_2) + [ ] Optional bool choice (OPT_BOOL_CHOICE) + + Enter a symbol/choice name, "load_config", or "write_config" (or press CTRL+D to exit): BOOL + Value for BOOL (available: n, y): n + ... + +I'm not currently interested in implementing a (more usable) menuconfig myself, but all the infrastructure +for a great one should be there if you want to give it a go. I'll help you out with any questions you might +have. + Sample ``make iscriptconfig`` session ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- cgit v1.2.3 From 8ab791266df0269d37905fd3cc21c62b24f88688 Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 12:09:20 +0100 Subject: Fix typo --- README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index 237df8c..9244de8 100644 --- a/README.rst +++ b/README.rst @@ -150,7 +150,7 @@ Examples Example scripts ~~~~~~~~~~~~~~~ -The ``examples/`` subdirectory contains some simple example. Among these are the following ones: +The ``examples/`` subdirectory contains some simple example scripts. Among these are the following ones: - `allnoconfig.py `_, `allnoconfig_simpler.py `_, and `allyesconfig.py `_ implement ``make allnoconfig`` and ``make allyesconfig`` in various ways. Demonstrates menu tree walking and value setting. -- cgit v1.2.3 From 9232538670d503c491230dd4e6ef619e1479314c Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 12:14:43 +0100 Subject: Add separate section for real-world examples --- README.rst | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index 9244de8..03908eb 100644 --- a/README.rst +++ b/README.rst @@ -202,6 +202,21 @@ I'm not currently interested in implementing a (more usable) menuconfig myself, for a great one should be there if you want to give it a go. I'll help you out with any questions you might have. +Real-world examples +~~~~~~~~~~~~~~~~~~~ + +These use the older Kconfiglib 1 API, which was a bit clunkier and not as general (functions instead of properties, and no direct access to the menu structure or expressions): + +- `genboardscfg.py `_ from `Das U-Boot `_ generates some sort of legacy board database by pulling information from a newly added Kconfig-based configuration system (as far as I understand it :). + +- `gen-manual-lists.py `_ generated listings for an appendix in the `Buildroot `_ manual. (The listing has since been removed.) + +- `SConf `_ builds an interactive configuration interface (like ``menuconfig``) on top of Kconfiglib, for use e.g. with `SCons `_. + +- `kconfig-diff.py `_ -- a script by `dubiousjim `_ that compares kernel configurations. + +- Originally, Kconfiglib was used in chapter 4 of my `master's thesis `_ to automatically generate a "minimal" kernel for a given system. Parts of it bother me a bit now, but that's how it goes with old work. + Sample ``make iscriptconfig`` session ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -286,18 +301,6 @@ The following log should give some idea of the functionality available in the AP CONFIG_STACKTRACE_SUPPORT=y CONFIG_MMU=y ... - -* The `examples/ `_ directory contains simple example scripts. See the documentation for how to run them. - -* `genboardscfg.py `_ from `Das U-Boot `_ generates some sort of legacy board database by pulling information from a newly added Kconfig-based configuration system (as far as I understand it :). - -* `gen-manual-lists.py `_ generated listings for an appendix in the `Buildroot `_ manual. (The listing has since been removed.) - -* `SConf `_ builds an interactive configuration interface (like ``menuconfig``) on top of Kconfiglib, for use e.g. with `SCons `_. - -* `kconfig-diff.py `_ -- a script by `dubiousjim `_ that compares kernel configurations. - -* Originally, Kconfiglib was used in chapter 4 of my `master's thesis `_ to automatically generate a "minimal" kernel for a given system. Parts of it bother me a bit now, but that's how it goes with old work. Test suite ---------- -- cgit v1.2.3 From 2294a0d3822062ef42b073b1b482b0856a06313b Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 12:24:20 +0100 Subject: Add a separate section with real-world examples Add the esp-idf script too. --- README.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index 03908eb..d9777b1 100644 --- a/README.rst +++ b/README.rst @@ -205,12 +205,14 @@ have. Real-world examples ~~~~~~~~~~~~~~~~~~~ -These use the older Kconfiglib 1 API, which was a bit clunkier and not as general (functions instead of properties, and no direct access to the menu structure or expressions): +These use the older Kconfiglib 1 API, which was clunkier and not as general (functions instead of properties, no direct access to the menu structure or properties, uglier ``__str__()`` output): - `genboardscfg.py `_ from `Das U-Boot `_ generates some sort of legacy board database by pulling information from a newly added Kconfig-based configuration system (as far as I understand it :). - `gen-manual-lists.py `_ generated listings for an appendix in the `Buildroot `_ manual. (The listing has since been removed.) +- `gen_kconfig_doc.py `_ from the `esp-idf `_ project generates documentation from Kconfig files. + - `SConf `_ builds an interactive configuration interface (like ``menuconfig``) on top of Kconfiglib, for use e.g. with `SCons `_. - `kconfig-diff.py `_ -- a script by `dubiousjim `_ that compares kernel configurations. -- cgit v1.2.3 From a1ae4e38503eb8a55712b6cd1f9d958035b0fd38 Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 12:29:51 +0100 Subject: Clean up the test suite section a bit --- README.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index d9777b1..4f85da7 100644 --- a/README.rst +++ b/README.rst @@ -313,12 +313,13 @@ The test suite is run with $ python(3) Kconfiglib/testsuite.py -(`pypy `_ works too, and is much speedier.) +(`pypy `_ works too, and is much speedier for everything except ``allnoconfig.py``/``allnoconfig_simpler.py``/``allyesconfig.py``, where it doesn't have time to warm up since +the scripts are run via ``make scriptconfig``.) -The test suite must be run from the top-level kernel directory. It requires that the git -repository has been cloned into it and that ``makefile.patch`` has been applied. +The test suite must be run from the top-level kernel directory, and requires that the git +repository has been cloned into it and that the makefile patch has been applied. -**NOTE: Some tests currently overwrite .config in the kernel root, so make sure to back it up.** +**NOTE: The test suite overwrites .config in the kernel root, so make sure to back it up.** The test suite consists of a set of selftests and a set of compatibility tests that compare (character for character) configurations generated by Kconfiglib with @@ -332,9 +333,8 @@ when a fix was added to Kconfig that's also mirrored in Kconfiglib This is harmless, and only counts as a fail since the test suite compares literal output from the kconfig version that's bundled with the kernel. -Kconfiglib is much faster than the test suite would indicate. Most of the time -is spent waiting around for ``make`` or the C utilities. Adding some multiprocessing -to the test suite would make sense. +A lot of the time is spent waiting around for ``make`` and the C utilities (which need to reparse all the +Kconfig files for each defconfig test). Adding some multiprocessing to the test suite would make sense. Notes ----- -- cgit v1.2.3 From b028c41014921cde824ed3b3208143e2369f9c63 Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Tue, 7 Nov 2017 12:40:04 +0100 Subject: Clean up the test suite section a bit more --- README.rst | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) (limited to 'README.rst') diff --git a/README.rst b/README.rst index 4f85da7..f08eafd 100644 --- a/README.rst +++ b/README.rst @@ -313,19 +313,20 @@ The test suite is run with $ python(3) Kconfiglib/testsuite.py -(`pypy `_ works too, and is much speedier for everything except ``allnoconfig.py``/``allnoconfig_simpler.py``/``allyesconfig.py``, where it doesn't have time to warm up since -the scripts are run via ``make scriptconfig``.) +`pypy `_ works too, and is much speedier for everything except ``allnoconfig.py``/``allnoconfig_simpler.py``/``allyesconfig.py``, where it doesn't have time to warm up since +the scripts are run via ``make scriptconfig``. -The test suite must be run from the top-level kernel directory, and requires that the git -repository has been cloned into it and that the makefile patch has been applied. +The test suite must be run from the top-level kernel directory. It requires that the +Kconfiglib git repository has been cloned into it and that the makefile patch has been applied. **NOTE: The test suite overwrites .config in the kernel root, so make sure to back it up.** The test suite consists of a set of selftests and a set of compatibility tests that -compare (character for character) configurations generated by Kconfiglib with -configurations generated by ``scripts/kconfig/conf`` for a number of cases. You -might want to use the "speedy" option; see -`testsuite.py `_. +compare configurations generated by Kconfiglib with +configurations generated by the C tools, for a number of cases. See +`testsuite.py `_ +for the available options. You might want to use the "speedy" option to speed things +up a bit. The test suite might fail for a few configurations for kernels older than April 2016, when a fix was added to Kconfig that's also mirrored in Kconfiglib @@ -333,8 +334,9 @@ when a fix was added to Kconfig that's also mirrored in Kconfiglib This is harmless, and only counts as a fail since the test suite compares literal output from the kconfig version that's bundled with the kernel. -A lot of the time is spent waiting around for ``make`` and the C utilities (which need to reparse all the -Kconfig files for each defconfig test). Adding some multiprocessing to the test suite would make sense. +A lot of time is spent waiting around for ``make`` and the C utilities (which need to reparse all the +Kconfig files for each defconfig test). Adding some multiprocessing to the test suite would make sense +too. Notes ----- -- cgit v1.2.3 From cafb421105f3acfa4266649d4466386bc0ac81d3 Mon Sep 17 00:00:00 2001 From: Ulf Magnusson Date: Wed, 8 Nov 2017 08:17:43 +0100 Subject: Explain changes from Kconfiglib 1 to Kconfiglib 2 I wonder if there's a nicer way to get an italic link with RST... --- README.rst | 7 ++++ kconfiglib-2-changes.txt | 106 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+) create mode 100644 kconfiglib-2-changes.txt (limited to 'README.rst') diff --git a/README.rst b/README.rst index f08eafd..2d7a985 100644 --- a/README.rst +++ b/README.rst @@ -4,6 +4,13 @@ Overview -------- +*This is version 2 of Kconfiglib, which is not backwards-compatible with +Kconfiglib 1. For a summary of changes between Kconfiglib 1 and Kconfiglib 2, +see* |changes|_. + +.. _changes: kconfiglib-2-changes.txt +.. |changes| replace:: *kconfiglib-2-changes.txt* + Kconfiglib is a Python 2/3 library for scripting and extracting information from `Kconfig `_ diff --git a/kconfiglib-2-changes.txt b/kconfiglib-2-changes.txt new file mode 100644 index 0000000..3cdcb2d --- /dev/null +++ b/kconfiglib-2-changes.txt @@ -0,0 +1,106 @@ +This is a summary of the changes between Kconfiglib 1 and Kconfiglib 2. + +The Kconfiglib 1 API was designed to solve a particular problem, before I even +knew that I'd release it as a library. I was also hazy on many details of +Kconfig internals when I first designed it. + +Bolting things like implicit menus and symbol introspection onto the Kconfiglib +1 API would have been awkward, so I decided to redo things the right way. + +API changes: + + - Read-only fields now use attributes instead of get_*() methods, which is + less clunky to work with. + + A few attributes use @property magic internally, but are still efficient to + access due to caching. + + - Tristate values are now represented as 0, 1, 2 instead of "n", "m", "y". + This is a small loss in readability in a few spots, but the advantages far + outweigh the disadvantages. Tristate values can now be compared directly, + and tristate n is naturally false. ("n" < "m" < "y" is false even + lexicographically, which is awkward.) + + The original design was an attempt to keep the value format consistent for + all types of symbols (as far as I can remember). + + - Much improved value API. Instead of get_upper/lower_bound(), is_assignable(), + etc., there's now just a single .assignable attribute, which returns a + sorted tuple with the currently assignable values (that would be reflected + in the value of the symbol). This corresponds to the selections available + in the menuconfig interface. + + For example, sym.assignable might be (0, 2) for a visible, non-selected + bool symbol. + + - Many seldom-used APIs, like get_ref_locations(), were removed. They can be + implemented manually in a much more general and powerful way with newly + added functionality (see e.g. examples/find_symbol.py). + + - Printing a Symbol or other Kconfig object now returns its representation in + Kconfig syntax instead of the ugly Kconfiglib 1 format. + + - Symbols and other Kconfig objects now have a __repr__() that returns useful + information (name, type, value, user value, prompt, location, etc.) + + - __str__() and __repr__() are deliberately implemented with just public + APIs, which indirectly ensures that all Kconfig object properties can be + fetched separately. + + - Probably lots of other stuff I forgot to mention. + + +New features: + + - Direct access to the menu tree + + Kconfiglib 1 represented the menu tree with nested lists, which works + poorly for e.g. implicit menus. Kconfiglib 2 uses a separate user-visible + MenuNode class that corresponds directly to 'struct menu' in the C + implementation, storing the same fields. + + The Menu and Comment classes were removed. They're represented as plain + menu nodes with node.item set to MENU and COMMENT, respectively. + + - Direct access to symbol properties and their conditions + + Expressions use a simple tuple-based format (think Lisp). + + Kconfiglib 1 stored constant (quoted) symbols as plain Python strings. This + makes expressions awkward to work with due to inconsistent value types. + Kconfiglib 2 uses constant Symbol instances for quoted symbols, like the C + implementation. + + - Better 'choice' handling + + In Kconfiglib 1, choices magically changed mode when values were assigned + to choice symbols, which was meant to make .config loading just work. In + Kconfiglib 2, the mode must be assigned manually (though most choices can + only be in n or y mode anyway and take care of themselves). + + Choices now have an .assignable attribute too, which makes the API + perfectly consistent between symbols and choices. The choice mode acts as + an upper bound on the visibility of the choice symbols (the choice is + simply propagated to property conditions, which makes this automatic). + + - Faster parsing code + + The helper classes were merged into the Kconfig (new name for Config) + class. This removes a lot of temporary object creation and passing around + of values. The 4.14 x86 Kconfigs now parse in about a second on a Core + i7-2600k (or much faster than that with PyPy if multiple configurations are + loaded). + + - Better invalidation algorithm + + Internal reorganization allowed for a better invalidation algorithm to be + used, which stops redundant invalidation much earlier and gives a huge + speedup for scripts that set a lot of values. The non-Kconfig-parsing + overhead is often irrelevant now. allyesconfig.py runs in about 1.5 seconds + when run via 'make scriptconfig', including the 'make' overhead. + + - Probably lots of other stuff I forgot to mention. + + +In general, Kconfiglib 2 is just a better and more general Kconfiglib 1, with +some old mistakes cleaned up. -- cgit v1.2.3