thelinuxlink.net

Posted: **Wed May 16, 2007 7:53 pm**

I would like to start a thread that tells about the hours of time you spend trying to figure out why your python app doesnt work at all. To start the thread off, here is what I did last night. I am writing an app that will convert mythtv videos to dvd, and burn it. So in this app I need to run some system commands. I used the function called popen3('command') which works for a command that will be executed right away and you dont have to worry about it. Well the commands I was using stuff like dvdauthor, mkisofs and mpeg2desc, all take time. So I run the script and it finishes right away but my iso has 0 bytes in it, I reran the app again, same thing, I did this about 5 more times and all of a sudden my system became real slow.........I did a ctrl+alt+f1 went to console and did 'ps -a' and found that there were a ton of instances of the apps I was running in my script running......Which leads me here, I cant believe I used a function that would not check to see if a command was finished before moving on to the rest of the app!
Ohh well live and learn I guess. I would love to hear if anyone else has any stories like this, it may help me feel better

Posted: **Fri May 18, 2007 8:55 pm**

I am usually a more 'lazy' or just social person, I pressure the community to bring what I need. I want to think that I leverage the community.

So usually I go into the IRC rooms and talk with developers and experts and ask everything that I can possibly think of until I have an undrestandment. This usually help me work with people and code.

However I dont think is the usual case, working with many techie guys I learn that must just DO RTFM and they learn on their own.

Posted: **Sat May 19, 2007 7:23 am**

I already mentioned mine earlier, but thought I'd bring it up here. It's pretty amazing that after years of using linux, I still managed to not RTFM.

I was trying to use pexpect to send a password to mysqldump, because mysqldump doesn't take the password from stdin and I wanted to know how to use pexpect.

The pexpect module is pretty simple, and a nice way to send commands to the shell, but basically I was trying to do something like this:

Code: Select all

child = pexpect.spawn(command)    #start the mysqldump command
child.expect('Enter password:')   #Expect the "Enter password:" prompt
sql = child.sendline(password)    #Save the text that comes after sending the password

I thought sql = child.sendline(password) seemed pythonic enough, but obviously I was way off base. After you send the password, you need to make another call to the expect method, then use one of the pexpect objects (or attributes?), in this case, child.before, though there's a child.after as well.

Code: Select all

child = pexpect.spawn(command)    #start the mysqldump command
child.expect('Enter password:')   #Expect the "Enter password:" prompt
child.sendline(password)          #Send the password
child.expect(pexpect.EOF)         #Expect the "End of File"
sql = child.before

#The pexpect module takes care of saving the text that came before the end of line, just need to use the "before" method to save it

After that I just used the open method to write the sql to a file, and I had the backup I wanted. There's a good chance I could do things better, and I realized recently that I was using popen to bzip the file, when there's a more pythonic way to do so.

I like to see code, so I hope others don't mind my request that people post some code with their problems and solutions on here when they mention their mistakes.

Posted: **Sun May 20, 2007 2:41 pm**

Vogateer wrote: I was trying to use pexpect to send a password to mysqldump, because mysqldump doesn't take the password from stdin and I wanted to know how to use pexpect.

Yeah it does. I have backup cripts running that look like this:
mysqldump -u<username> -p<password> <dbname> > filename.sql

Just take note that there is *no whitespace* between the -u and <username> and the -p and the <password>. This is how I have been doing my sequal data backups for years now.

Posted: **Mon May 21, 2007 2:00 pm**

Thanks, Linc, I knew I had forgotten some little thing like that, I did put the space in between the switch and the password.

The script works as I have it, but python uses close to 100% of the CPU when using pexpect to handle this. I'm assuming the pexpect module—written in pure python—wasn't meant to handle that amount of data with it's methods. So I know how to use pexpect, which may come in handy later, but it's not meant to be used for sql dumps.

Now, I seem to remember having issues attempting to use bash pipes and redirects in python, so I'll have to look into that when I get home.

Posted: **Mon May 21, 2007 3:36 pm**

why not use os.sys module or the MySQLdb.

Reading and asking at #python and #mysql up in freenode I found a way to backup the mysql database from php. Python shouldn't be that different since the key is using LOAD DATA command wihtin the mysql prompt.

In other words you can use LOAD DATA and SELECT INTO OUTFILE executed from a mysql shell than from a system shell (BASH).

Code: Select all

<?php
include 'config.php';
include 'opendb.php';

$tableName  = 'mypet';
$backupFile = 'backup/mypet.sql';
$query      = "SELECT * INTO OUTFILE '$backupFile' FROM $tableName";
$result = mysql_query($query);

include 'closedb.php';
?>

This is to import:

Code: Select all

<?php
include 'config.php';
include 'opendb.php';

$tableName  = 'mypet';
$backupFile = 'mypet.sql';
$query      = "LOAD DATA INFILE 'backupFile' INTO TABLE $tableName";
$result = mysql_query($query);

include 'closedb.php';
?>

Python might work something similar:

Code: Select all

....
table = "mypet"
backupfile = "mypet.sql"
db = MySQLdb.connect("host machine", "dbuser", "password", "dbname")
cursor = db.cursor()
sql = "SELECT * INTO OUTFILE " $table " FROM " $backupfile
cursor.execute(sql)
data = cursor.fetchone()
db.close()

thelinuxlink.net

Good times learning Python!

Good times learning Python!