Saving a lot of photos

[eluser]Ignacio[/eluser]
Ok, you have a website with 2M users, each user have it thumbnail, right? So, you have 2M images to save in your server/s.

Now, my question is about the names of the thumbnails/photos and sizes too.

You can see something like this over the net:
http://vthumb.ak.facebook.com/vthumb-ak-...0_1031.jpg
http://a606.ac-images.myspacecdn.com/ima...88e555.jpg
http://photos1.hi5.com/0026/985/229/tlwis7985229-01.jpg
http://en.netlogstatic.com/p/tt/012/097/12097680.jpg
http://userserve-ak.last.fm/serve/50/2052205.jpg

Ok, you know what I mean, right? What are those numbers, stuff, I don't know, I'm always use ID.FILENAMETYPE (ie: file.jpg)

There're any script for CI for doing this well like PRO sites?
There're anybody with ideas?

Thanks for sharing, Ignacio

[eluser]Référencement Google[/eluser]
Yes, take a look at the Upload library, there is the preference option 'encrypt_name' that you will have to set to TRUE and that will make what you need.

[eluser]Ignacio[/eluser]
Thanks elitemedia, I have solved the name of the photo with that feature, now... what about those folders?

[eluser]Sean Murphy[/eluser]
Ignacio, setting the 'encrypt_name' option to true will not do what you're asking for. This is a rather complex and case specific topic.

The sites you mentioned above store their assets across several severs and in directory structures each unique to themselves. The reason they do so is because there is a limit to how many files can be stored in a single directory, or served from a single server.

Basically, you'll have to craft your own solution for your situation. If you still want to do so and would like help, I can go into more detail.

[eluser]James Pax[/eluser]
just do this after upload:

1) save the photos details in a db (ex. MySQL) (also location)

2) then name the photo after the photo's ID

you will obtain a similar result, facebook photos are probably stored like that

if you want to encrypt

1) save the photos details in a db (like above) including the "encrypted name field"

I'm not sure what kind of benefit encryption over db ID gives, but if you feel more secure with it then head with the second option

[eluser]Ignacio[/eluser]
Sean Murphy, please, I want to see your details, I'm thinking you know what I'm looking for.

darksky0, hehe, I did that a lot of times, but I have the same question of you, "I’m not sure what kind of benefit encryption over db ID gives", hehe.

[eluser]Edemilson Lima[/eluser]
You could make a script to create a new directory to save the images when the current one have, for example, 5 thousand images. The directory name could start with 00000 and increase by 1. What you need to do is record somewhere in your database the number of the last current directory and the number of files it have. As every file have a related row in a table, you must have an SMALLINT field to save the directory number and another field to save the encrypted file name (or a serial file number). So, that way you can get the file and keep the directories with a good number of files. If in the future you need to separate the images among different web servers, you can do this by moving the folders to the servers and creating symbolic links to them.

[eluser]Sean Murphy[/eluser]
How many media and application servers would you like me to use in my example?

[eluser]Ignacio[/eluser]
Edemilson Lima, THANKS, that is really helpful.

Sean Murphy, I'm exited to see your solution, but your asking how many servers, is that important? I don't know,... I really want to see what are you doing. I have 3 servers. 1: webapp, 2: media files.

[eluser]Sean Murphy[/eluser]
I'm just crafting one possible solution (of dozens) off the top of my head. So please, take it with a little salt, and the understanding that this is just an example and would obviously need some tweaking and more detail.

In this configuration we'll store all versions (sizes) of the same file on the same server.

Some key points to consider in setting something like this up are:
-hot/warm/cold storage
Media server local discs will be hot. AWS (S3) will be used for hot or warm storage.
-failover
Should a media server go down, files will be served from S3.
-load balancing
Load balancing will focus primarily on storage capacity and will be handled when storing new files.
-performance
Media servers should run a thin web server such as lighttpd or non-threaded Apache. It may also be wise to use a CDN.

In the CI config (on the app servers) you'll put an array containing a list of media servers. This array is used to determine where NEWLY uploaded files are stored. You might write a function for selecting a server that does a little dumb load balancing based on media server capacity, etc.

When a new file is uploaded you'll select a media server from those available, do any image resizing or editing needed, and transfer the files to the media server. You'll also transfer the same files to S3. You might do something like Edemilson Lima mentioned for your directory structure, and make sure to store in the DB which media server the file lives on.

It would probably be wise to have a function for building asset urls for use in your application views.

When a request comes in for something like http://media1.example.com/123456789/123456789.jpg it will go to the load balancer and, if the media1 server is up, the file will be served directly from there. If media1 is down, the LB redirects the request to an application server, which will then serve the file from S3.

I hope that helps at least some. There's a lot I glazed over. Let me know if you want more detail on something.