missing tempo-query binary

Summary

tempo-query is unable to start due to a missing GRPC storage plugin binary (/tmp/tempo-query).

Steps to reproduce

docker run registry1.dso.mil/ironbank/opensource/grafana/tempo-query:2.2.2

What is the current bug behavior?

tempo-query fails to start

What is the expected correct behavior?

tempo-query should start and the Jaeger web interface should be present

Relevant logs and/or screenshots

2023/09/12 13:41:57 maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
{"level":"info","ts":1694526117.7793791,"caller":"flags/service.go:119","msg":"Mounting metrics handler on admin server","route":"/metrics"}
{"level":"info","ts":1694526117.780806,"caller":"flags/service.go:125","msg":"Mounting expvar handler on admin server","route":"/debug/vars"}
{"level":"info","ts":1694526117.7825572,"caller":"flags/admin.go:129","msg":"Mounting health check on admin server","route":"/"}
{"level":"info","ts":1694526117.783179,"caller":"flags/admin.go:143","msg":"Starting admin HTTP server","http-addr":":16687"}
{"level":"info","ts":1694526117.7838852,"caller":"flags/admin.go:121","msg":"Admin server started","http.host-port":"[::]:16687","health-status":"unavailable"}
{"level":"fatal","ts":1694526117.8157434,"caller":"./main.go:109","msg":"Failed to init storage factory","error":"grpc-plugin builder failed to create a store: error attempting to connect to plugin rpc client: fork/exec /tmp/tempo-query: no such file or directory","stacktrace":"main.main.func1\n\t./main.go:109\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.7.0/command.go:940\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.7.0/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.7.0/command.go:992\nmain.main\n\t./main.go:172\nruntime.main\n\truntime/proc.go:250"}

Possible fixes

I believe the issue was introduced with this commit, wherein the line:

COPY --from=builder --chown=root:root /tmp/tempo-query /tmp/tempo-query

was removed.

This line may have been removed due to this pipeline failing.

Removing that line resolves the container build issues in the pipeline, but it prevents the copy of the tempo-query binary and thus, prevents tempo-query from running properly.

The reason that pipeline failed is due to this upstream commit, wherein Tempo developers decided to place the binary in /tempo-query instead of /tmp/tempo-query to accommodate read-only file system K8s environments.

A fix could be adding that line back (but slightly different to accommodate the upstream changes):

COPY --from=builder --chown=root:root /tempo-query /tempo-query

And modifying the corresponding environment variable, from:

GRPC_STORAGE_PLUGIN_BINARY=/tmp/tempo-query

to:

GRPC_STORAGE_PLUGIN_BINARY=/tempo-query

Tasks

• Bug has been identified and corrected within the container

Please read the Iron Bank Documentation for more info