{"id":1955,"date":"2009-11-12T23:41:21","date_gmt":"2009-11-13T06:41:21","guid":{"rendered":"http:\/\/multimedia.cx\/eggs\/?p=1955"},"modified":"2009-11-12T23:55:25","modified_gmt":"2009-11-13T06:55:25","slug":"process-runner-redux","status":"publish","type":"post","link":"https:\/\/multimedia.cx\/eggs\/process-runner-redux\/","title":{"rendered":"Process Runner Redux"},"content":{"rendered":"

Pursuant to yesterday’s conundrum<\/a> of creating a portable process runner in Python for FATE<\/a> that can be reliably killed when exceeding time constraints, I settled on a solution. As Raymond Tau reminded us<\/a> in the ensuing discussion, Python won’t use a shell to launch the process if the program can supply the command and its arguments as a sequence data structure. I knew this but was intentionally avoiding it. It seems like a simple problem to break up a command line into a sequence of arguments– just split on spaces. However, I hope to test metadata options eventually which could include arguments such as ‘-title “Hey, is this thing on?”‘ where splitting on spaces clearly isn’t the right solution.<\/p>\n

I got frustrated enough with the problem that I decided to split on spaces anyway. Hey, I control this system from top to bottom, so new rule: No command line arguments in test specs will have spaces with quotes around them. I already enforce the rule that no sample files can have spaces in their filenames since that causes trouble with remote testing. When I get to the part about testing metadata, said metadata will take the form of ‘-title “HeyIsThisThingOn?”‘ (which will then fail to catch myriad bugs related to FFmpeg’s incorrect handling of whitespace in metadata arguments, but this is all about trade-offs).<\/p>\n

So the revised Python process runner seems to work correctly on Linux. The hangaround.c program simulates a badly misbehaving program by eating the TERM signal and must be dealt with using the KILL signal. The last line in these examples is a tuple containing return code, stdout, stderr, and CPU time. For Linux:<\/p>\n

\r\n$ .\/upr.py \r\n['.\/hangaround', '40']\r\nprocess ID = 2645\r\ntimeout, sending TERM\r\ntimeout, really killing\r\n[-9, '', '', 0]\r\n<\/pre>\n
The unmodified code works the same on Mac OS X:<\/p>\n
\r\n$ .\/upr.py\r\n['.\/hangaround', '40']\r\nprocess ID = 94866\r\ntimeout, sending TERM\r\ntimeout, really killing\r\n[-9, '', '', 0]\r\n<\/pre>\n
Now a bigger test: Running the upr.py script on Linux in order to launch the hangaround process remotely on Mac OS X via SSH:<\/p>\n
\r\n$ .\/upr.py \r\n['\/usr\/bin\/ssh', 'foster-home', '.\/hangaround', '40']\r\nprocess ID = 2673\r\ntimeout, sending TERM\r\n[143, '', '', 50]\r\n<\/pre>\n
So that’s good… sort of. Monitoring the process on the other end reveals that hangaround is still doing just that, even after SSH goes away. This occurs whether or not hangaround is ignoring the TERM signal. This is still suboptimal.<\/p>\n
It would be possible to open a separate SSH session to send a TERM or KILL signal to the original process… except that I wouldn’t know the PID of the remote process. Or could I? I’m open to Unix shell magic tricks on this problem since anything responding to SSH requests is probably going to be acceptably Unix-like. I would rather not go the ‘killall ffmpeg’ route because that could interfere with some multiprocessing ideas I’m working on.<\/p>\n
Here’s a brute force brainstorm: When operating in remote-SSH mode, prefix the command with ‘ln -s ffmpeg ffmpeg-<unique-key>’ and then execute the symbolic link instead of the main binary. Then the script should be able to open a separate SSH session and execute ‘killall ffmpeg-<unique-key>’ without interfering with other processes. Outlandish but possibly workable.<\/p>\n","protected":false},"excerpt":{"rendered":"
Getting closer to a bulletproof process runner for FATE<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[101,55],"tags":[],"class_list":["post-1955","post","type-post","status-publish","format-standard","hentry","category-fate-server","category-python"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t