Step-by-Step: Diagnosing Ruby EventMachine reactor block due to synchronous I/O operations on Google Cloud Servers
Identifying the Root Cause: Synchronous I/O in EventMachine
EventMachine is a popular Ruby library for building asynchronous and event-driven network applications. Its core strength lies in its non-blocking I/O model, managed by an event loop (the reactor). When this reactor becomes blocked, the entire application grinds to a halt, leading to unresponsiveness and dropped connections. A common culprit for reactor blocking, especially in cloud environments like Google Cloud Platform (GCP), is the inadvertent execution of synchronous I/O operations within EventMachine callbacks or handlers. These operations, such as blocking network requests to external services or synchronous file system access, prevent the reactor from processing other events, effectively freezing the application.
Leveraging GCP Tools for Initial Diagnosis
Before diving deep into application logs, it’s crucial to leverage GCP’s monitoring and logging capabilities to get a high-level view of the problem. Look for:
- Cloud Monitoring Metrics: Check CPU utilization, network traffic, and request latency for your Compute Engine instances or GKE nodes. Spikes in CPU or latency, coupled with a decrease in network throughput, can indicate a stalled application.
- Cloud Logging: Search for application logs that coincide with periods of unresponsiveness. Look for error messages, timeouts, or any unusual patterns. Filter by your application’s service name and relevant time ranges.
Application-Level Diagnostics: Tracing the Block
Once you suspect a reactor block, the next step is to pinpoint the exact synchronous operation causing the issue. This often involves instrumenting your Ruby application to provide more granular insights into its execution flow.
1. Enabling Detailed EventMachine Logging
EventMachine itself can be configured to log more verbosely. While not always sufficient on its own, it can provide clues about which callbacks are being invoked and when.
To enable debug logging, you can set the EM_DEBUG environment variable:
export EM_DEBUG=1 # Then run your EventMachine application
This will flood your logs with EventMachine’s internal state changes. You’ll need to sift through this to find patterns around the time of unresponsiveness. Look for repeated entries related to a specific connection or handler, especially if they don’t seem to be yielding control back to the reactor.
2. Using Thread Dumps for Deadlocks and Long-Running Operations
When an EventMachine reactor is blocked, the main thread is occupied. A thread dump can reveal what that thread is doing. In Ruby, you can generate a thread dump by sending a QUIT signal to the process. This is often done via the kill command.
# Find the process ID (PID) of your Ruby application pgrep -f 'your_eventmachine_app.rb' # Send the QUIT signal to the PID kill -QUIT
The thread dump will be printed to standard error (stderr). Analyze this output for the main thread (often the one running the EventMachine reactor). Look for calls to blocking I/O operations like:
Net::HTTP.get,Net::HTTP.post, etc. (synchronous HTTP requests)TCPSocket#connect,TCPSocket#read,TCPSocket#write(blocking socket operations)File.read,File.write(synchronous file I/O)- Database driver methods that are not explicitly configured for non-blocking I/O.
If you see these within the stack trace of the main thread, you’ve likely found your culprit. For example, a thread dump might show something like this for the main thread:
"main" prio=5 tid=0x00007f8b4c001000 nid=0x1234 runnable [0x00007f8b4d000000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:196) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:135) at java.io.BufferedInputStream.read1(BufferedInputStream.java:1348) at java.io.BufferedInputStream.read(BufferedInputStream.java:1750) - locked <0x00000007a1b2c3d4> (a java.io.BufferedInputStream) at java.io.FilterInputStream.read(FilterInputStream.java:66) at org.jruby.internal.runtime.io.JavaIOAdapter$1.read(JavaIOAdapter.java:100) at org.jruby.RubyIO.read(RubyIO.java:1203) at org.jruby.RubyIO.read(RubyIO.java:1185) at org.jruby.RubyIO.read(RubyIO.java:1169) at org.jruby.RubyIO.read(RubyIO.java:1153) at org.jruby.RubyProc.call(RubyProc.java:228) at org.jruby.RubyProc.call(RubyProc.java:214) at org.jruby.RubyMethod.call(RubyMethod.java:133) at org.jruby.RubyKernel.require(RubyKernel.java:1064) at org.jruby.RubyKernel.require(RubyKernel.java:1050) at org.jruby.RubyKernel.require(RubyKernel.java:1038) at org.jruby.RubyKernel.require(RubyKernel.java:1026) at org.jruby.RubyKernel.require(RubyKernel.java:1014) at org.jruby.RubyKernel.require(RubyKernel.java:1002) at org.jruby.RubyKernel.require(RubyKernel.java:990) at org.jruby.RubyKernel.require(RubyKernel.java:978) at org.jruby.RubyKernel.require(RubyKernel.java:966) at org.jruby.RubyKernel.require(RubyKernel.java:954) at org.jruby.RubyKernel.require(RubyKernel.java:942) at org.jruby.RubyKernel.require(RubyKernel.java:930) at org.jruby.RubyKernel.require(RubyKernel.java:918) at org.jruby.RubyKernel.require(RubyKernel.java:906) at org.jruby.RubyKernel.require(RubyKernel.java:894) at org.jruby.RubyKernel.require(RubyKernel.java:882) at org.jruby.RubyKernel.require(RubyKernel.java:870) at org.jruby.RubyKernel.require(RubyKernel.java:858) at org.jruby.RubyKernel.require(RubyKernel.java:846) at org.jruby.RubyKernel.require(RubyKernel.java:834) at org.jruby.RubyKernel.require(RubyKernel.java:822) at org.jruby.RubyKernel.require(RubyKernel.java:810) at org.jruby.RubyKernel.require(RubyKernel.java:798) at org.jruby.RubyKernel.require(RubyKernel.java:786) at org.jruby.RubyKernel.require(RubyKernel.java:774) at org.jruby.RubyKernel.require(RubyKernel.java:762) at org.jruby.RubyKernel.require(RubyKernel.java:750) at org.jruby.RubyKernel.require(RubyKernel.java:738) at org.jruby.RubyKernel.require(RubyKernel.java:726) at org.jruby.RubyKernel.require(RubyKernel.java:714) at org.jruby.RubyKernel.require(RubyKernel.java:702) at org.jruby.RubyKernel.require(RubyKernel.java:690) at org.jruby.RubyKernel.require(RubyKernel.java:678) at org.jruby.RubyKernel.require(RubyKernel.java:666) at org.jruby.RubyKernel.require(RubyKernel.java:654) at org.jruby.RubyKernel.require(RubyKernel.java:642) at org.jruby.RubyKernel.require(RubyKernel.java:630) at org.jruby.RubyKernel.require(RubyKernel.java:618) at org.jruby.RubyKernel.require(RubyKernel.java:606) at org.jruby.RubyKernel.require(RubyKernel.java:594) at org.jruby.RubyKernel.require(RubyKernel.java:582) at org.jruby.RubyKernel.require(RubyKernel.java:570) at org.jruby.RubyKernel.require(RubyKernel.java:558) at org.jruby.RubyKernel.require(RubyKernel.java:546) at org.jruby.RubyKernel.require(RubyKernel.java:534) at org.jruby.RubyKernel.require(RubyKernel.java:522) at org.jruby.RubyKernel.require(RubyKernel.java:510) at org.jruby.RubyKernel.require(RubyKernel.java:498) at org.jruby.RubyKernel.require(RubyKernel.java:486) at org.jruby.RubyKernel.require(RubyKernel.java:474) at org.jruby.RubyKernel.require(RubyKernel.java:462) at org.jruby.RubyKernel.require(RubyKernel.java:450) at org.jruby.RubyKernel.require(RubyKernel.java:438) at org.jruby.RubyKernel.require(RubyKernel.java:426) at org.jruby.RubyKernel.require(RubyKernel.java:414) at org.jruby.RubyKernel.require(RubyKernel.java:402) at org.jruby.RubyKernel.require(RubyKernel.java:390) at org.jruby.RubyKernel.require(RubyKernel.java:378) at org.jruby.RubyKernel.require(RubyKernel.java:366) at org.jruby.RubyKernel.require(RubyKernel.java:354) at org.jruby.RubyKernel.require(RubyKernel.java:342) at org.jruby.RubyKernel.require(RubyKernel.java:330) at org.jruby.RubyKernel.require(RubyKernel.java:318) at org.jruby.RubyKernel.require(RubyKernel.java:306) at org.jruby.RubyKernel.require(RubyKernel.java:294) at org.jruby.RubyKernel.require(RubyKernel.java:282) at org.jruby.RubyKernel.require(RubyKernel.java:270) at org.jruby.RubyKernel.require(RubyKernel.java:258) at org.jruby.RubyKernel.require(RubyKernel.java:246) at org.jruby.RubyKernel.require(RubyKernel.java:234) at org.jruby.RubyKernel.require(RubyKernel.java:222) at org.jruby.RubyKernel.require(RubyKernel.java:210) at org.jruby.RubyKernel.require(RubyKernel.java:198) at org.jruby.RubyKernel.require(RubyKernel.java:186) at org.jruby.RubyKernel.require(RubyKernel.java:174) at org.jruby.RubyKernel.require(RubyKernel.java:162) at org.jruby.RubyKernel.require(RubyKernel.java:150) at org.jruby.RubyKernel.require(RubyKernel.java:138) at org.jruby.RubyKernel.require(RubyKernel.java:126) at org.jruby.RubyKernel.require(RubyKernel.java:114) at org.jruby.RubyKernel.require(RubyKernel.java:102) at org.jruby.RubyKernel.require(RubyKernel.java:90) at org.jruby.RubyKernel.require(RubyKernel.java:78) at org.jruby.RubyKernel.require(RubyKernel.java:66) at org.jruby.RubyKernel.require(RubyKernel.java:54) at org.jruby.RubyKernel.require(RubyKernel.java:42) at org.jruby.RubyKernel.require(RubyKernel.java:30) at org.jruby.RubyKernel.require(RubyKernel.java:18) at org.jruby.RubyKernel.require(RubyKernel.java:6) at org.jruby.Ruby.eval(Ruby.java:861) at org.jruby.Ruby.eval(Ruby.java:824) at org.jruby.Ruby.eval(Ruby.java:808) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:744) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:737) at org.jruby.embed.ScriptingContainer.runScriptlet(Script