Linode.com Forum

[1st]

Hi there, I quite recently bought a linode, and was wondering how to get this script working...

Here is the PHP code
Code:
<?
set_time_limit(0);
mysql_connect("localhost","root","[myrootpassword]");
mysql_select_db("db1");
$words=file("words.txt");

foreach ($words as $word) {
$word = mysql_escape_string(rtrim($word));
$query = mysql_query("INSERT INTO `des` (word) VALUES ('{$word}')");
}
echo "worked.";
?>


For every line in there, it's suppose to add it into the database. It works for small files, however, the file I am working with is around 2gb
So I updated the memory allocation in php.ini yet, when I run this I get a blank page... any suggestions?

Thank in advance,
a new user

[Xan]

It could take quite some time to insert a 2GB file line by line. Are you sure it's not actually working, until your Web server times out? Try running it without going through the Web server.

As an aside, I think one of the biggest problems with PHP is that it encourages quoting rather than binding variables when passing commands to SQL. One mistake and you've got an injection attack. http://sqlrelay.sourceforge.net/sqlrelay/programming/phppeardb.html#bindvars

[1st]

Ah, I wouldn't worry about injection attacks, the page is basically isolated from the rest of the content, with no input executed, except that which is escaped from the inserting method. With few even knowing how to access the front page.

Well, I set it up so it wouldn't time out, and it doesn't even load. Just generates a blank page, with no queries being executed. Ideas?

By the way, when I try this on another server, it inserts the queries... but, it stops at 50000, for quota reasons.

[kenny]

1st wrote: Hi there, I quite recently bought a linode, and was wondering how to get this script working...

Here is the PHP code
Code:
<?
$words=file("words.txt");
?>

It works for small files, however, the file I am working with is around 2gb

file reads an entire file into an array.. so you are trying to read a 2gb file into memory. Try fgets and insert one line at a time.

Code:

<?
set_time_limit(0);
mysql_connect("localhost","root","[myrootpassword]");
mysql_select_db("db1");

$handle = fopen("words.txt", "r");
while (!feof($handle)) {
    $word = fgets($handle);
    $word = mysql_escape_string(rtrim($word));
    $query = mysql_query("INSERT INTO `des` (word) VALUES ('{$word}')");
}
echo "worked.";
?>


kenny