Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support recursive dicts #3

Merged
merged 6 commits into from
Oct 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ Features include:
- Both role (inline) and directive (paragraph level) support.
- Theoretical Markdown support via `myst_parser
<https://github.com/executablebooks/MyST-Parser>`__.
- Support nested mappings within yaml using `.` to drill into keys.
- Partial key lookup support. You only need to specify some substring
within the key to get the substitution. If it matches more than one,
an error will be thrown.


Demo
Expand Down Expand Up @@ -193,6 +197,9 @@ Currently there are two Sphinx variables defined:
need for both, but it provides more flexible configuration for
integration to your build system.

..
* If ``partial_match_substitution`` set to true, then only a partial
string match to the key will result in a substitution.


Development and maintenance
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
Sphinx
docutils!=0.15
PyYAML
jsonpath-ng
14 changes: 13 additions & 1 deletion sphinx_ext_substitution/get_replacements.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,21 @@ def _load_yaml(fname, substitutions):
if yaml is None:
raise RuntimeError("The yaml module is needed (python-yaml) to get definitions from yaml files.")
data = yaml.load(open(fname), Loader=yaml.SafeLoader)
_strip_whitespace_recursively(data, substitutions)
print("substitutions")
print(substitutions)


def _strip_whitespace_recursively(data, substitutions):
"""Save this dict's keys as substitutions, recurse with prefix into sub-dicts."""
for key, value in data.items():
if key not in substitutions:
substitutions[key] = value.strip()
if isinstance(value, str):
substitutions[key] = value.strip()
elif isinstance(value, dict):
substitutions[key] = _strip_whitespace_recursively(value, data[key])
return substitutions


def load_substitutions(config):
"""Load substitutions from disk. Cache results to SUBSTITUTIONS.
Expand Down
21 changes: 20 additions & 1 deletion sphinx_ext_substitution/substitution.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
from sphinx.locale import _
import sphinx.util.nodes

from jsonpath_ng import jsonpath, parse

from .get_replacements import get_substitutions

class sub(nodes.Admonition, nodes.Element):
Expand Down Expand Up @@ -69,7 +71,8 @@ def sub_role(name, rawtext, text, lineno, inliner,
if id_ in subs:
replacement = subs[id_]
else:
replacement = None
replacement = _jsonpath_lookup(id_, subs)

if replacement:
replacement = replacement.replace('\x00`', '`')

Expand Down Expand Up @@ -122,6 +125,20 @@ def sub_role(name, rawtext, text, lineno, inliner,
return content, messages


def _jsonpath_lookup(id_, subs):
try:
expression = parse(id_)
except:
return None
matches = [match.value for match in expression.find(subs)]
if len(matches) == 0:
replacement = None
elif all([m == matches[0] for m in matches]):
replacement = matches[0]
else:
raise UserWarning(f'Different replacements found for {id_}... matches found: {matches}')
return replacement


class SubDirective(Directive):
required_arguments = 1
Expand All @@ -142,6 +159,8 @@ def run(self):
# Get the replacement value, don't use it yet
if id_ in subs:
replacement = subs[id_]
if not isinstance(replacement, str):
raise UserWarning(f"Yields a non-string substitution for {id_}... resulted in {replacement}")
replacement = statemachine.StringList(
replacement.splitlines(), source='file')
else:
Expand Down
31 changes: 30 additions & 1 deletion sphinx_ext_substitution/test_basic.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,36 @@ def doc1_original():
"""Document with substitute_mode=original"""
return doc(build='_build-original', opts="-D substitute_mode=original")


@pytest.fixture(scope='session')
def doc1_partial_match():
"""Document with partial_match_substitution=true"""
return doc(build='_build-partial-match', opts="-D partial_match_substitution=true")

def testFindJsonPath():
from jsonpath_ng import jsonpath, parse
# Sample JSON data
data = {
"name": "John",
"address": {
"city": "New York",
"state": {
"code": "NY",
"name": "New York"
}
}
}

# Compile the JSONPath expression
expression = parse("$..code")

# Use the expression to find matches in your data
matches = [match.value for match in expression.find(data)]

print(matches) # Output: ['NY']

def test_map_id(doc1_default):
index = doc1_default['index']
assert "A13-id-substitute" in index

def test_role(doc1_default):
index = doc1_default['index']
Expand Down
14 changes: 13 additions & 1 deletion testdata/proj/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,18 @@ No replacement (preformatted): :sub:`A7-id: \`\`A7-original\`\``

Replacement (preformatted): :sub:`A8-id: \`\`A8-original\`\``

Map Replacement :sub:`$.A12-id.A13-id:\`\`A13-original\`\``

Map Replacement :sub:`$.plant.lfom.rowN:\`\`rown-og\`\``

Map Replacement :sub:`$..lfom.rowN:\`\`rown-og\`\``

Map Replacement :sub:`$..rowN:\`\`rown-og\`\``

Map Replacement :sub:`$..rowB:\`\`rown-og\`\``

Map Replacement :sub:`rowb:\`\`rown-og\`\``

Directive with no replacement
-----------------------------

Expand All @@ -52,4 +64,4 @@ Directive with replacement
Directive with no content
-------------------------

.. sub:: A12-id
.. sub:: $.A12-id.A13-id
9 changes: 9 additions & 0 deletions testdata/substitutions/one-yaml/subs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,12 @@ A11-id: |
A11.1-substitute

*A11.2-substitute*

A12-id:
A13-id: A13-id-substitute
A14-id: A14-id-substitute

plant:
lfom:
rowN: '5'
rowB: '5 cm'