Presentación
Vídeo
Transcripción
Extracto de la transcripción automática del vídeo realizada por YouTube.
- Afternoon, everyone. Thank you for hangin' tough, until the last session before the closing keynote, and comin' along. My name's Jason Clark, I work as an engineer at New Relic, and that informs a little bit of what we're gonna talk about today. I'm going to start off, and this talk's kind of sectioned into a couple of different examples, that you may be able to relate to.
The first one is, has anyone here ever had a process, they've got some Ruby program that's running, and it's stopped. Like it's still running, it's still there, but it's not doing what it's supposed to do. It's deadlocked in some fashion, it's hung up somewhere and you don't really know what's going on, so you look, the process is there, but it's hardly churnin' any CPU.
There's just no detectable activity. You look at the log file, and the log file's just got nothin'. Imagine that you're in this scenario, and just to up the ante a little, recently at this particular place, maybe your coworkers had been talking about adding some multi-threading support, into this service, you know? It'll make it way faster, if we put some threads in there! And all of a sudden you have this deep and abiding dread, because you think you might have a deadlock.
Your Ruby process is stuck and you have no idea why and you don't know where this is happening in the code. Well that's the sort of situation where GDB can come to your rescue. So GDB stands for the Gnu Debugger. It is actually a very old tool, the first version of it was released in 1986, but it is a C-level debugger, among other things there's lots of extensions to it, but it allows you to dig into the native layer of code, below where your Ruby is executing.
Now, we're at RubyConf, like why am I talking about a C-debugger, why do we care about this? Well, the majority of us use MRI. That's reference implementation for Ruby, Matz's Rubi, and the truth is that Matz actually prefers that we call it C Ruby. The Ruby VM that you are executing your code on is written in C.
And so GDB as a tool that has visibility to that layer will let us see what's happening underneath. We can take a peek at what the Ruby VM is actually doing for us, to try to solve these sorts of problems. So this is intended as a gentle introduction, you're not expected to know any C, but I'm gonna step you through how you can use GDB as a tool to understand what's happening in these processes underneath you.
So, with our stuck process, there's a couple of conventions that we'll follow in this talk and in the output. So when there's a $ at the beginning of a line, that indicates a command that we're gonna issue at our terminal. So at our terminal, we're gonna say gdb -p and then give it the process ID of the process that we care about, this stuck process running on our system.
Now, GDB's gonna throw a fair amount of output. It's gonna tell you it's attaching to things, it may spit out stuff about symbols that it's loading, but then eventually it lands you at this. So when there is a (gdb) in this as well, that is a prompt within GDB itself where you can issue commands.
So GDB has taken the process that we were running, it's paused its execution, and it's now asking us, well, what do you wanna do? The first thing that you might want to know is where is this code that's executing? I seem to be stopped, where am I actually stopped at? So, the natural thing to ask for, is to ask GDB for a backtrace.
If you do this in actual practice, the output may be quite a bit longer than what I've demonstrated here. I don't know if anyone here has ever used GDB, but it will generate some pretty big stack traces and a lot of output on the screen. So some of this is lightly formatted and truncated.
But this isn't something for us to be scared of, right? This looks a little more complicated, but it's basically the same thing as this, which I'm pretty sure anyone who's programmed in Ruby for more than five minutes has seen. This is just a stack trace, this shows us the methods and the files that we've stepped through.
So let's try and break down what GDB's shown us and see where this is similar to what we know and what we can glean out of it. First off, because the output is pretty long and because of some of how it displays, it gives us these nice line numbers, 'cause some of those lines might wrap.
So those are frames that we can go through, where we've stepped in our program, and each of them has a function name associated with it. You can think of these a lot like our Ruby methods. These would be methods that you would find in your code that you would be able to identify and locate.
In fact, a lot of the time, if you've got the appropriate debugging symbols available, it will even tell you where in the source code that came from. So here we see something in our Unix system dependencies, but then thread. c, on the third one, that's actually a file in MRI itself, and this is sitting on line 4342 of that file.
So we could look and see exactly where this is executing. And in fact, that line is very pertinent to what we care about as well, because we're suspecting some sort of deadlock, right? Something that's stopping our process. Well, MRI methods, conventionally start with a RB underscore in the function name, and here we have RB underscore mutex lock.
So this is starting to smell even more like a deadlock, like we might have expected. When we, say, create a mutex object and ask it to lock itself in Ruby, this is the C function that's actually being run by the VM on our behalf. This is a little bit of a picture of what's going on in this process, but just seeing one thread being somewhere in a lock doesn't necessarily tell us everything we need to know.
So let's ask GDB what's going on elsewhere in the process. If we say info threads, GDB will give us a list of all of the threads that are in the current process, and show us the top level of where those are actually executing. So the threads are numbered and those numbers stay consistent across the lifetime of the thread, and like we saw in the backtrace, it gives us the function names.
Interestingly, here we can see this pthread cond wait, that was actually the top thing in the frame for that particular backtrace that we were paused on, and here we see that there are two separate threads which are both at that same line of code, waiting on a condition in the pthread library.
So this is giving us even more information, this is smelling more and more like our theory that we've got a deadlock is true, but we'd really like to see how we got to those locations as well. And GDB provides. With the thread apply command, you can select certain threads, and in this case I'm selecting all of the threads in the process, and ask GDB to run a specific command against those.
So we can take that backtrace that we took originally, and that ran for just the current thread that we were on, and instead ask GDB to show us all of the backtraces for all of the threads. And if we do this, we do in fact see that thread number three has that RB mutex lock, right where we had seen it before, where we were paused, and thread one is also paused at that exact same lock.
With this information in hand, we can go to our coworker who thought they were so smart, putting multi-treading into the thing, point out that they may well have introduced a deadlock, and hopefully they're smart enough to fix it. You know, your mileage may vary depending on who it is, and having done so, we wanna get out of GDB.
You type exit and, well, exit's not actually defined unfortunately, so we'll quit instead. Let's pause for a moment before we move on and just review a little bit of what we've talked about here. We've seen how you can use GDB and attach to a running process.
[ ... ]
Nota: se han omitido las otras 3.794 palabras de la transcripción completa para cumplir con las normas de «uso razonable» de YouTube.