COMMUNITY FORUM

Cache folder & number of files.A possible solution

Start
Prev
1
Next
End

Jim Fasoulas
Topic Author
Offline
New Member

10 years 3 months ago #129889 by Jim Fasoulas

Cache folder & number of files.A possible solution was created by Jim Fasoulas

Hello.

I'm redesigning a huge site that uses k2 and have the following problem.
The cache folder currently has 34.495 files. Those files where created in a period of 1 and a half year. And as the site took off more and more content gets uploaded each day.

Now size is not a problem for me as disk space is quite cheap nowadays. Thing is that at some point I'm going to hit a limit of maximum files on directory. Besides the obvious wall I'm about to hit there is also the problem of accessing those files taking considerable more time (100 ms more than other files).

This problem will actually brake the site I'm currently working on so I have to try and find a solution. I have something in mind but I wanted to run it by the developers and get some help on where I should look and also whats their view is on the problem.

As far as I know when you create a k2 item you give it a picture, K2 creates an md5 hash from the item id and creates the images with a file-name of that hash plus the size (_XS _S _M etc) in the cache folder.

When an item is displayed k2 checks the cache folder to see if a file with the proper name exists and displays it.

What I'm thinking of doing is this.

When an item is created check its creation date and create the files in /media/k2/items/cache/YEAR (ex. /media/k2/items/cache/2014)
When an items is displayed check its creation date and search for the file in /media/k2/items/cache/YEAR in staid of /media/k2/items/cache/

One problem that arises from the above solution is that if the creation date of an item changes from lets say 2014 to 2013 then that item will be displayed without an image but I'm willing to overlook that as that wont happen very often and if needed I can just re-upload an image.

So here are my questions.
Will that change have any impact in the site ?
And which are the files responsible for creating the image and displaying an image ?

Thank you in advance
Dimitris

Please Log in or Create an account to join the conversation.

Jim Fasoulas
Topic Author
Offline
New Member

10 years 3 months ago #129890 by Jim Fasoulas

Replied by Jim Fasoulas on topic Re: Cache folder & number of files.A possible solution

Proof of concept working on the administration side of K2 (when you edit an item)

Changes in the file administrator/components/com_k2/views/item/view.html.php

Original code :

		$date = JFactory::getDate($item->modified);
		$timestamp = '?t='.$date->toUnix();

		if (JFile::exists(JPATH_SITE.DS.'media'.DS.'k2'.DS.'items'.DS.'cache'.DS.md5("Image".$item->id).'_L.jpg'))
		{
			$item->image = JURI::root().'media/k2/items/cache/'.md5("Image".$item->id).'_L.jpg'.$timestamp;
		}

		if (JFile::exists(JPATH_SITE.DS.'media'.DS.'k2'.DS.'items'.DS.'cache'.DS.md5("Image".$item->id).'_S.jpg'))
		{
			$item->thumb = JURI::root().'media/k2/items/cache/'.md5("Image".$item->id).'_S.jpg'.$timestamp;
		}

Changed code:

		$date = JFactory::getDate($item->modified);
		// patch for cache folder
		// create a variable with the name of the folder as y2014y (2014 is the creation date)
		$year = JFactory::getDate($item->created);
		$year = $year->Format('Y');
		$year = 'y'.$year.'y';
		$timestamp = '?t='.$date->toUnix();

		
		if (JFile::exists(JPATH_SITE.DS.'media'.DS.'k2'.DS.'items'.DS.'cache'.DS.$year.DS.md5("Image".$item->id).'_L.jpg')) #check if the file exist in the folder y2014y
		{
			$item->image = JURI::root().'media/k2/items/cache/'.$year.'/'.md5("Image".$item->id).'_L.jpg'.$timestamp; #set item image with the path /media/k2/items/cache/y2014y/filename
		} elseif (JFile::exists(JPATH_SITE.DS.'media'.DS.'k2'.DS.'items'.DS.'cache'.DS.md5("Image".$item->id).'_L.jpg')) { #if the file doesnt exist check is its on on /cache/ (this is for items created before the patch)
			$item->image = JURI::root().'media/k2/items/cache/'.md5("Image".$item->id).'_L.jpg'.$timestamp;
		}

		//if (JFile::exists(JPATH_SITE.DS.'media'.DS.'k2'.DS.'items'.DS.'cache'.DS.md5("Image".$item->id).'_S.jpg'))
		if (JFile::exists(JPATH_SITE.DS.'media'.DS.'k2'.DS.'items'.DS.'cache'.DS.$year.DS.md5("Image".$item->id).'_S.jpg'))
		{
			//$item->thumb = JURI::root().'media/k2/items/cache/'.md5("Image".$item->id).'_S.jpg'.$timestamp;
			$item->thumb = JURI::root().'media/k2/items/cache/'.$year.'/'.md5("Image".$item->id).'_S.jpg'.$timestamp;
		} elseif (JFile::exists(JPATH_SITE.DS.'media'.DS.'k2'.DS.'items'.DS.'cache'.DS.md5("Image".$item->id).'_S.jpg')) {
			$item->thumb = JURI::root().'media/k2/items/cache/'.md5("Image".$item->id).'_S.jpg'.$timestamp;
		}

This only does checks to see if the file exists in /media/k2/items/cache/YEAR. It does not create the folder or the files in the folder. But I created the folder by hand and copied the files and it seems to be working .

Next step would be the actual creation of the files and the view on the front-end

Please Log in or Create an account to join the conversation.

Lefteris
Offline
Moderator

10 years 3 months ago #129891 by Lefteris

Replied by Lefteris on topic Re: Cache folder & number of files.A possible solution

Hi. The maximum number of files per directory depends on the file system of your server. In any case ,I don't think that you are going to hit it even in 100 years.... Your concept is correct but this requires hacking K2 so i would stay with the current solution.

JoomlaWorks Support Team
---
Please search the forum before posting a new topic :)

Please Log in or Create an account to join the conversation.

Start
Prev
1
Next
End