i am trying to store image in sql on server. Later i want to make these image available for download. Which approach is good:
By uploading image and saving path.
or
2.By converting image to base64 and then storing as BLOB type
I'd say it's a tradeoff, and it also depends a lot on your network connection, I'd go for keeping the images in the cloud and just keeping a key in the database.
It would shrink the disk usage a lot , possibly even allowing the entire database to be cached into ram, thus balancing the extra time the process would spend retrieving the relevant info from the network
If the security is main concern then store image in DB in blob, or if if it is not much sensitive then store it in specific directory then store it's path in DB.
Because DB storage give it extra security.
If we store it on directory then someone can access it if they have tried some trick to access the directory.
So, and if we store it on directory then proper handling of file should be done.
Because if someone upload file then again other person store it wit same name then possibility of file overwrite.
So you can do something like that take 2 fields in table one is original name and one is new name that have original name plus the timestamp of file upload and in directory store it with that new name.
and if you are storing it in DB then doesn't need this extra work.
And yes bifurcation of images in directory also tough, suppose in one year so many file will be stored in it so indexing plus accessing become cumbersome work.
So apply some mechanism so at some time of interval you can separate images.
Related
Extending this thread - I would just like to know why it's faster to retrieve files from a file system, rather than a MySQL database. If one were to benchmark the two to see which would retrieve the most data (multiple types of data) over 10 minutes - which one would win?
If a file system is truly faster, then why not just store everything in a file system and replace a database with csv or xml?
EDIT 1:
I found a good resource for alternate storage options for java
EDIT 2:
I'm looking for a Java API/Jar that has the functionality of a SQL Database Server Engine (or at least some of it) that uses XML for data storage (preferably). If you know of something, please leave a comment below.
At the end of the day the database does just store the data in the file system. It's all the useful stuff on top of just the raw data that makes you decide to use a database.
If you can replicate the functionality, scalability, robustness, integrity, etc, etc of a database system using CSV and still make it perform faster than a relational database then yes I'd suggest doing it your way.
It'd take you a few years to get there though.
Of course, relational systems are not the only way to store data. There are object-oriented database systems (db4o, InterSystems Cache) and document-based systems (RavenDB).
Performance is also relative to the style and volume of data you are working with and what you intend to do with it - I'm not going to even try and discuss that, it's too open ended.
I will also not start the follow on discussion: if memory is truly faster than the file system, why not just store everything in memory? :-)
This also seems similar to another question I answered a long while ago:
Is C# really slower than say C++?
Basically stuff isn't always done just for performance.
MySQL uses the file system the same as everything else on a computer. To retrieve a single piece of data, or a table of data, there is no faster way that directly from the file system. MySQL would just be a small bit of overhead added to that file system pull.
If you need to do some intelligent selecting, match some rows, or filter that data, MySQL is going to do that faster than most other options. The database server provides you calculation and data manipulation power that a filesystem can't.
When you have mixed/structured data, a DBMS is the only solution. For eg. try to get the people's name, surname and country for all your customers stored into your DB, but only those born in 1981 and living in Rome. If you have this data into files on the filesystem, how do you easily get only the required data without scanning all your files and how do you join returned data?
A DBMS give you much more than that.
Many DBMS store data into files.
This abstraction layer will make you retrieve data in a very easily, standard and structured way.
The difference is in how the desired data is located.
In a file system, locating the desired data means searching through all existing data until you find it.
Databases provide indexing which results in locating the desired data almost immediately (within ~12 comparisons) regardless of the amount of data.
What we want is an indexed file system - lucky for us, we have them. They are called databases.
I need some inputs and suggestions from you. I have a very huge database which has around 2000 records having some information.
is it good to have another database having key value pair pointing to that huge database or XML file is enough?
Having 2000 records is not huge. And its better to use SQLite for data operations rather than using xml file, because an xml file with 2000 pairs will make the processing slow and is resource wasting. Better use SQLite for such requirements.
The web app has to let a user browse a csv file of around 100'000 lines.
We have to show the contents of the file on the screen so the user can see what has been loaded.
Then the user will perform some actions on the screen (e.g. add their details)
We need to store the file in database and associate it with the user record
The contents and the format of the file will change per user.
Therefore, I cannot create a static table in MS SQL DB for it.
Do you recommend loading this file as a blob in MS SQL 2008 DB or should I load the file, create a plain class (in C#) object that matches the file, serialise it and then store it as the xml file. Later if user wants to see it, I can read the xml string from DB, deserialise it to the object, write logic that takes the object and creates a csv file?
Is there another approach? Is storing as xml string better than storing as blob?
You could use the FILESTREAM data type in SQL Server 2008. Behind the scenes the actual file is stored on the file system but you get fully transactional access to it and the way it is actually stored is completely transparent.
Hi I am new to mysql and java aswell. I want to store a jpeg files and hash values of small chunks of file. I have stored the hash values of small chunks(100s in number) and now want to store the jpeg file also against these small chunks. My question is do I need to store the file again and again for each record or is it possible to save file once and link it to the records related to the file? if so then also please guide me that how can I do it?
You can save the file on the machine and store its path in the database.
Suppose you are having 1 field (say imagePath) in the table which takes "varchar" data. You can store the path of the image there and retrieve the image at runtime. By doing this, you can avoid saving same file multiple times. However, it will override the images having same name but different data. For that you have to use Primary key to append to the name of the file. I hope it will help you to understand.
I must admit I'm not entirely sure what you want to do, but if understand you correctly this is what I would do.
One table (say tblJpgs) with data about the jpg-file, maybe the path and the filename as suggested by Naved and additionally a description and whatever can be useful info about the file. In this table there will be one row per jpg-file and each row will have a unique id.
Then you will have another table (say tblChunks) for all the chunks. There should be a column for connecting each row with a tblJpgs.id. Then there is of course a column with the chunk itself. In this table there will be one row for each chunk, but there will be many rows for one jpg-file.
In this way you will only save the information in one place, which is very central in database structures. Storing the jpg-file or the path and filename for it in each row with the chunks would be against this fundamental database structure and should therefore be avoided.
I am working currently on a web project where users can create image galleries and upload pictures. Optionally they can mark pictures as private so that nobody else can look at them.
Now i am not sure how to properly implement the protection mechanism. Of course i have some ideas but none of them seems to be optimal.
Here is one of my ideas:
Create a table for user images:
image_key (PK) | user_id | public_image (boolean)
the picture will be saved on the harddisk using the iamge_key and can be accessed via http by an url looking like this:
http://www.myCompany.com/images/image_key
a servlet will be mapped to the url path images, the key will be extracted, a stream to the file on the harddisk will be openend and the picutre will be streamed.
additionally there will be a reverse proxy in front of the servlet container to provide some caching.
The issue with this solution is that my serlvet would have to go to the database and check if the image with the given key is public or not.
My question:
Can this be done without hitting the database? (some fancy ideas)
Can someone provide a better solution to store and keep track of the pictures?
How would a solution look like where besides public and private pictures also some pictures are shared to firends only?
Please note that this question is not about storing pictures in a database or somewhere else but concerns access rights management of binary resources in a web application environment.
If the DB table is properly indexed and you're using a connection pool, then hitting the DB is cheap. I would just keep it as is. You can at highest have a copy of the table in a Map in the application scope. But this may eat too much server memory. If you're using an ORM framework like JPA/Hibernate, you could also just turn the second level cache on to delegate the caching to the ORM. It will generally do its job very well.
As to the client side caching, you'd like to have a short expire time (1 hour?) and provide a doHead() in the servlet which in turn does basically just the same as doGet() but then without writing any bit to the response body. You would also like to check for If-None-Match and If-Last-Modified headers in the servlet if the client supplied them. You can find here a basic example of a servlet which does that.
My question: Can this be done without
hitting the database? (some fancy
ideas)
Yup, you can do it without hitting the database. We've got something similar and just wanted to put something quick in place.
The user is marking the resource private or public when he's uploading it.
We do something very simple:
public resources have a "tinyurl like" URL, say: abcd.org/aZ3tD (part of the point of the very short tinyurl-link thing is so that people who want to cut/paste/twitter it don't have to use an additional layer of bit.ly or tinyurl)
private resources aren't meant to be shared nor archived, so users don't care about a URL looking like: abcd.org/private/da499c3314e2fdce6a10a8b985489671971c187d
The first part of that URL is the user's ID.
So only the user da499c3314e2 (which must be logged in) can access resource fdce6a10a8b985489671971c187d
You asked for some way to do this without hitting the DB, this should work...
To avoid having to go to the database so often, how about the following URL patterns:
1. http://www.mycompany.com/images/<user_id>/public/<image_key>
2. http://www.mycompany.com/images/<user_id>/private/<image_key>
3. http://www.mycompany.com/images/<user_id>/shared/<image_key>
For 1, obviously no DB lookup is required - the image can be served to anybody immediately. For 2, your servlet would have to check that the ID of the active user matches the user_id in the request - again, hopefully no DB lookup required, just a check of a session variable.
The only case in which a DB call would be needed is 3, to check the relationship between the requesting user and the user who owns the image.
Of course, you'll need to be very careful about caching to ensure that your cache doesn't serve up private or shared images to unauthorised users...
Another alternative can be to store such information in image metadata.
Image metadata API: http://download-llnw.oracle.com/javase/1.4.2/docs/api/javax/imageio/metadata/package-summary.html
You can find related example:
http://download-llnw.oracle.com/javase/1.5.0/docs/guide/imageio/spec/apps.fm5.html
Write dpi metadata to a jpeg image in Java