Keyword

Fatal error on Import Joomla Content - memory allocation

  • Tim Coldwell
  • Tim Coldwell's Avatar Topic Author
  • Offline
  • New Member
More
14 years 1 month ago #87508 by Tim Coldwell
I have ~25,000 joomla articles to import.  I got to about 2,000 and got this error:
Fatal error: Out of memory (allocated 89653248) (tried to allocate 1945 bytes) in/home/globalpr/public_html/k2gpm/libraries/joomla/database/database/mysql.php on line 462

Any ideas?

Please Log in or Create an account to join the conversation.

More
14 years 1 month ago #87509 by william white
I would try breaking the import into smaller chunks. As there is not a way to do this with default k2, i would first try:
phpmyadmin - export to csv, bring the file to local machine
use xcel or local mysql, or other database program to split the file into multiple csv files
then i would use www.individual-it.net/en/Joomla/ to import the articles to items.
If your articles are divided into groups by joomla section and/or category that may be a way to use phpmyadmin to break them into chunks

Please Log in or Create an account to join the conversation.

  • Tim Coldwell
  • Tim Coldwell's Avatar Topic Author
  • Offline
  • New Member
More
14 years 1 month ago #87510 by Tim Coldwell
Thanks. I follow your suggested method, however, even the compressed Akeeba .gpa backup file is > 350MB for this site. How big do you think the CSV download file might be? Until the K2 Import content feature is improved (automatic forced splitting into manageable chunks?) adding the K2 component to an existing largish Joomla 1.5 site seems, at first sight, impracticable at best.

Tim

William White said:I would try breaking the import into smaller chunks. As there is not a way to do this with default k2, i would first try: phpmyadmin - export to csv, bring the file to local machine
use xcel or local mysql, or other database program to split the file into multiple csv files
then i would use www.individual-it.net/en/Joomla/ to import the articles to items.
If your articles are divided into groups by joomla section and/or category that may be a way to use phpmyadmin to break them into chunks

Please Log in or Create an account to join the conversation.

More
14 years 1 month ago #87511 by william white
There may be a simple solution, but tracking it down might be difficult. The environment can have many settings that will affect whats going on. My idea was to "avoid" the Joomla environment and overhead partially by using the server php/mysql environment only at first to get the data. Then let Joomla enviornment with its built in overhead take a smaller piece at a time.
I have seen/written desktop apps that run in a database environment where you get an odbc connection to the remote table, then copy it to a local file, process it locally to change the structrure and reorganize it in the needed format, then insert it into the "other" in this case k2 tables to get the desired result.
Arthur @ the link i supplied should probably already know about the limitations of his extension

Please Log in or Create an account to join the conversation.

  • Tim Coldwell
  • Tim Coldwell's Avatar Topic Author
  • Offline
  • New Member
More
14 years 1 month ago #87512 by Tim Coldwell
I understand that in the Joomla + K2 environment that the way to avoid duplicates of imported articles is to archive those that have already been imported. Maybe someone could work out a way to run a process where joomla content is pulled (and replaced/deleted) in smallish chunks from a pseudo temp archive to build the import to K2 until it's finished without hitting memory allocation errors. I'm not expressing this concept too clearly but hopefully the gist is decipherable:-) Meanwhile I'll try to contact Arthur.

William White said:There may be a simple solution, but tracking it down might be difficult. The environment can have many settings that will affect whats going on. My idea was to "avoid" the Joomla environment and overhead partially by using the server php/mysql environment only at first to get the data. Then let Joomla enviornment with its built in overhead take a smaller piece at a time. I have seen/written desktop apps that run in a database environment where you get an odbc connection to the remote table, then copy it to a local file, process it locally to change the structrure and reorganize it in the needed format, then insert it into the "other" in this case k2 tables to get the desired result.
Arthur @ the link i supplied should probably already know about the limitations of his extension

Please Log in or Create an account to join the conversation.

More
14 years 1 month ago #87513 by william white
Try on a test environment, but i am pretty sure that k2 import buttion "will" create a copy of the items in k2 if you click it twice.
I think that the import joomla was a means to get k2 up and running quickley and not to routinely import items...unless you import once, delete all joomla content, add joomla articles, import into k2, delete them now so as not to duplicate on next import.

Please Log in or Create an account to join the conversation.

  • Tim Coldwell
  • Tim Coldwell's Avatar Topic Author
  • Offline
  • New Member
More
14 years 1 month ago #87514 by Tim Coldwell
Now I can't even get to Article Manager in Joomla :-( Fatal error indeed.

Fatal error: Out of memory (allocated 103284736) (tried to allocate 46 bytes) in /home/globalpr/public_html/k2gpm/libraries/joomla/database/database/mysql.php on line 462

Tim

William White said:Try on a test environment, but i am pretty sure that k2 import buttion "will" create a copy of the items in k2 if you click it twice. I think that the import joomla was a means to get k2 up and running quickley and not to routinely import items...unless you import once, delete all joomla content, add joomla articles, import into k2, delete them now so as not to duplicate on next import.

Please Log in or Create an account to join the conversation.

More
13 years 4 months ago #87515 by madcat
Have you solved the error, i'm face the same issue

Please Log in or Create an account to join the conversation.


Powered by Kunena Forum