Skip to content

Use semantic JSON logging in production#284

Open
vangberg wants to merge 1 commit into
mainfrom
production-json-logging
Open

Use semantic JSON logging in production#284
vangberg wants to merge 1 commit into
mainfrom
production-json-logging

Conversation

@vangberg

@vangberg vangberg commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Let's discuss this in-person maybe.

Structured JSON log with the Semantic Logger gem.

This will change the format of the messages logged to production.log to JSON, e.g.:

{"host":"Mac","application":"seqcode-registry","environment":"development","timestamp":"2026-06-18T08:53:26.417321Z","level":"debug","level_index":1,"pid":23874,"thread":"puma srv tp 005","duration_ms":0.4759998992085457,"duration":"0.476ms","named_tags":{"request_id":"aca6012d-4fc0-4fa4-9ef8-086377cb11bf"},"name":"ActiveRecord","payload":{"sql":"SELECT COUNT(*) FROM \"names\" WHERE \"names\".\"status\" = $1","binds":{"status":15},"allocations":254,"cached":null}}
{"host":"Mac","application":"seqcode-registry","environment":"development","timestamp":"2026-06-18T08:52:38.871463Z","level":"debug","level_index":1,"pid":23874,"thread":"puma srv tp 003","named_tags":{"request_id":"54192b6f-d025-4fed-8f32-2ba51396a7b6"},"name":"Rack","message":"Started","payload":{"method":"GET","path":"/api/v1/names/1.json","ip":"127.0.0.1"}}
{"host":"Mac","application":"seqcode-registry","environment":"development","timestamp":"2026-06-18T08:52:38.878342Z","level":"fatal","level_index":5,"pid":23874,"thread":"puma srv tp 003","file":"/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/concurrent-ruby-1.3.6/lib/concurrent-ruby/concurrent/atomic/thread_local_var.rb","line":93,"named_tags":{"request_id":"54192b6f-d025-4fed-8f32-2ba51396a7b6"},"name":"Rails","exception":{"name":"ActionController::RoutingError","message":"No route matches [GET] \"/api/v1/names/1.json\"","stack_trace":["/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/actionpack-6.1.7.10/lib/action_dispatch/middleware/debug_exceptions.rb:33:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/web-console-4.2.1/lib/web_console/middleware.rb:132:in `call_app'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/web-console-4.2.1/lib/web_console/middleware.rb:28:in `block in call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/web-console-4.2.1/lib/web_console/middleware.rb:17:in `catch'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/web-console-4.2.1/lib/web_console/middleware.rb:17:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/actionpack-6.1.7.10/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/rails_semantic_logger-4.20.0/lib/rails_semantic_logger/rack/logger.rb:53:in `call_app'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/rails_semantic_logger-4.20.0/lib/rails_semantic_logger/rack/logger.rb:26:in `block in call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/semantic_logger-4.18.0/lib/semantic_logger/base.rb:202:in `block in tagged'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/semantic_logger-4.18.0/lib/semantic_logger/semantic_logger.rb:395:in `named_tagged'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/semantic_logger-4.18.0/lib/semantic_logger/base.rb:209:in `tagged'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/rails_semantic_logger-4.20.0/lib/rails_semantic_logger/rack/logger.rb:26:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/actionpack-6.1.7.10/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/actionpack-6.1.7.10/lib/action_dispatch/middleware/request_id.rb:26:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/rack-2.2.23/lib/rack/method_override.rb:24:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/rack-2.2.23/lib/rack/runtime.rb:22:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/activesupport-6.1.7.10/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/actionpack-6.1.7.10/lib/action_dispatch/middleware/executor.rb:14:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/actionpack-6.1.7.10/lib/action_dispatch/middleware/static.rb:24:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/rack-2.2.23/lib/rack/sendfile.rb:127:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/actionpack-6.1.7.10/lib/action_dispatch/middleware/host_authorization.rb:148:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/rack-cors-2.0.2/lib/rack/cors.rb:102:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/webpacker-5.4.4/lib/webpacker/dev_server_proxy.rb:25:in `perform_request'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/rack-proxy-0.7.7/lib/rack/proxy.rb:87:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/railties-6.1.7.10/lib/rails/engine.rb:539:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/puma-5.6.9/lib/puma/configuration.rb:252:in `call'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/puma-5.6.9/lib/puma/request.rb:77:in `block in handle_request'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/puma-5.6.9/lib/puma/thread_pool.rb:340:in `with_force_shutdown'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/puma-5.6.9/lib/puma/request.rb:76:in `handle_request'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/puma-5.6.9/lib/puma/server.rb:443:in `process_client'","/Users/harryvangberg/.asdf/installs/ruby/3.3.9/lib/ruby/gems/3.3.0/gems/puma-5.6.9/lib/puma/thread_pool.rb:147:in `block in spawn_thread'"]}}

Stdout logs are still human-readable:

2026-06-18 10:53:26.365013 D [23874:puma srv tp 005] {request_id: aca6012d-4fc0-4fa4-9ef8-086377cb11bf} Rack -- Started -- {:method=>"GET", :path=>"/", :ip=>"127.0.0.1"}
2026-06-18 10:53:26.367874 D [23874:puma srv tp 005] {request_id: aca6012d-4fc0-4fa4-9ef8-086377cb11bf} ApplicationController -- Processing #main
2026-06-18 10:53:26.377091 D [23874:puma srv tp 005] {request_id: aca6012d-4fc0-4fa4-9ef8-086377cb11bf} (6.677ms) ActiveRecord -- {:sql=>"SELECT \"names\".\"nomenclatural_type_id\" FROM \"names\" WHERE \"names\".\"status\" IN ($1, $2, $3, $4) AND \"names\".\"redirect_id\" IS NULL AND \"names\".\"nomenclatural_type_type\" = $5", :binds=>{:status=>[0, 15, 20, 25], :nomenclatural_type_type=>"Genome"}, :allocations=>36, :cached=>nil}

Grafana can parse JSON so it can be used to filter and group. In particular, this should make it possible to create a dashboard which groups and sums exceptions, which would be really helpful in tracking down/noticing errors.

@vangberg vangberg requested a review from lmrodriguezr June 18, 2026 08:55
# config.logger = ActiveSupport::TaggedLogging.new(Syslog::Logger.new 'app-name')
config.rails_semantic_logger.format = :json

if ENV['RAILS_LOG_TO_STDOUT'].present?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not seem to be in use anywhere. We could remove the whole block I think.

@lmrodriguezr

Copy link
Copy Markdown
Member

Hi @vangberg
We didn't discuss this PR, but it looks good to me. If you don't have any reservations with it, it should be good to go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants