SKIP UPDATE CHECK Handle pipe escaping differently for strings and objects
General MR
Summary
Recently, an update to loki's README.md
broke the nightly doc compiler pipeline. After investigation, I discovered that |
characters need to be escaped differently for objects than for strings. The error occurs due to this line.
This change preserves the existing behavior for all types other than objects. For objects, pipes will be escaped with 2 backslashes so that the resulting output is compatible with docs_compiler's usage of json.loads
.
While investigating, I found 2 locations where a |
character was present within a multi-line string (as opposed to just being at the start). I went alphabetically through the packages and stopped at keycloak as I believe I found enough samples to be certain that this MR won't be overly broad.
- The loki values.yaml that caused the pipeline failure.
- In loki, the
enterprise
entry contains theconfig
as part of an object.
- In loki, the
- In keycloak: https://repo1.dso.mil/big-bang/product/packages/keycloak/-/blob/main/chart/values.yaml?ref_type=heads#L168
- In keycloak,
affinity
is treated as a string.
- In keycloak,
Relevant logs/screenshots
Link to failing job: https://repo1.dso.mil/big-bang/team/tools/bb-docs-compiler/-/jobs/34866065
Error message:
────────────────────────────── loki@6.5.2-bb.0 ───────────────────────────────
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/builds/big-bang/team/tools/scheduled-build/docs_compiler/cli.py", line 278, in compiler
compile(bb, ref)
File "/builds/big-bang/team/tools/scheduled-build/docs_compiler/cli.py", line 142, in compile
write_values_md(f"docs/packages/{pkg}/values.md", values_table, pkg)
File "/builds/big-bang/team/tools/scheduled-build/docs_compiler/utils.py", line 73, in write_values_md
yaml.dump(json.loads(data["Default"]), indent=2).split(r"\n")
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Invalid \escape: line 1 column 254 (char 253)
Linked Issue
Upgrade Notices
N/A
Testing
I used this version of _templates.gotmpl
to regenerate the README.md
for both loki and keycloak.
In the case of loki, the output changed (expected behavior) and was compatible with docs_compiler
(verified by running docs_compiler
locally with a pair of branches -- 1 umbrella, 1 loki -- where the changed README.md
was committed).
In the case of keycloak, the output did not change (expected behavior).