What would be a scalable file upload/download system/database?
I'm building a website where users can login, upload images that are private, but truly private. I can't upload them to a map on the harddisk of a server, since that would not scale (what happend if we add more servers?) and it wouldn't be private since everyone could go:
http://127.372.171.33/images/private_picture.png
and download the file.
I am building the project in Play Framework (scala/java)
How do websites like flickr handle these kind of things? Do they put them in a database? And what kind of database would be suitable for this situation?
Thanks for help
I can't tell you how those big sites handle it but putting those images into a database might be one way.
Another way would be to put the files into a virtual filesystem that spans a cluster of servers or distribute them onto different servers and just don't make the directories that contain the images visible to the webserver. Thus nobody should be able to open the image just using the server and the path on that server.
To actually deliver the images you could them implement some streaming service that sends a bytestream to the browser for display (like the webservers would do as well). This service could first check the download permissions for the requested image.
Related
I've built an app where users can upload their avatars. I used the paperclip gem and everything works fine on my local machine. On Heroku everything works fine until server restart. Then every uploaded images disappear. Is it possible to keep them on the server?
Notice: I probably should use services such as Amazon S3 or Google Cloud. However each of those services require credit card or banking account information, even if you want to use a free mode. This is a small app just for my portfolio and I would rather avoid sending that information.
No, this isn't possible. Heroku's filesystem is ephemeral and there is no way to make it persistent. You will lose your uploads every time your dyno restarts.
You must use an off-site file storage service like Amazon S3 if you want to store files long-term.
(Technically you could store your images directly in your database, e.g. as a bytea in Postgres, but I strongly advise against that. It's not very efficient and then you have to worry about how to provide the saved files to the browser. Go with S3 or something similar.)
I'm currently developing a Tracking System with JavaFX and MySQL as database that is kept in a server. My application is used within a network and allows users to upload and download pictures and several types of document.
My question: What is the best way to send and retrieve files to a server within a network? And should I store those files in MySQL or just their paths in MySQL? If only files' paths, do I need FTP or other techniques? I need detail answer because it is my first time to develop such application.
Edit: I want to store data in a server. I build this application for client machines so that clients can keep documents in the server and they can access their files from any machines... I have no idea for how to transfer files from client machines to server. Please help me!
You should use socket programming like what is at following link:
http://www.rgagnon.com/javadetails/java-0542.html
.But if you are using Java7, Files is the best one instead of BufferedInputStream or FileInputStream (No extra library require):
/* You can get Path from file also: file.toPath() */
Files.copy(InputStream in, Path target)
Files.copy(Path source, OutputStream out)
Say I have an image file that is located in http://a.com/example.jpg
Then I have a network storage called http://b.com/
Using java, how can I move example.jpg directly to b.com?(Kind of like P2P??)
EDIT
I think I should explain a little bit more.
I want to move file example.jpg to b.com without downloading the file.
Normal approach will be
Open connection to a.com
Download example.jpg to my computer (either memory or disk)
Open a new connection to b.com
Upload the example.jpg to b.com
However, I think this involves unnecessary work.
What I want is this
Open connection to b.com
Without opening connection to a.com, upload example.jpg directly to b.com (I think this is the part where I thought of P2P)
Is there any way that I can do this?
There is no general way to do what you ask merely with HTTP. You would have to have specific support for such operations on the server hosting your network storage, such that you can request it to download URLs directly.
You can get the image using HttpClient from a.com and then upload it to b.com. How do you upload it b.com depends on what you have there.
To be able to do this without downloading the file you will need specific support on b.com. That specific support will be a service on b.com where you send a "download" request and the server will download the file itself from a.com.
I'm developing an application which copy PDF files to server. My swing application scan and label the documents which are scanned then copy to server. There are two roles doctypesA read and write,doctypesB read and write. I stored roles in database and handle this rights from my swing application. When user use my application there isn't any problem because ı can control, total 200 users approx. But my problem is that anyone who knows the link on server SEVER\MYDOCS\doctypesa\doc.pdf can see it. It is intranet application, how can i restrict folder in this situtation.
MY APP JAVA SWING, INTRANET
I consider to solve this problem, ı'll use encrypt/ decrypt my pdf files.
I am making a website where coworkers can share files with one another.
One user chooses a file on his computer.
He leaves his computer on and that web page open.
Other multiple users can download that specified file from his computer.
Specs:
There are no main servers. The one
user with the file, his computer is
the server,
persay.
It is all done a website, no program
for users to download.
Also, I guess what I mean by no main server is that I dont have to actually buy servers for large files. Basically I want to code a nice, no main server p2p network.
Could I use cirrus?
Would really appreciate any help.
Please & Thanks.
There is no way of implementing P2P without having some central peers facilitating P2P connections.
Moreover, websites are implemented with servers. If your peers are going to run a web service, they need to be localized on the web. You will some kind of central service to accomplish this.
is that I dont have to actually buy servers for large files.
You won't have to. You can serve .torrent files (usually a few kB) from any computer. You could use an open tracker such as http://openbittorrent.com/Open Bittorrent to "use" these bittorrent files. Your employees can then use a torrent client (Vuze, Utorrent, etc.) to download the files...
Another easy way would be using something like Opera Unite.
A web-based, decentralized file sharing the way you mention it, would most likely be very hard to create and maintain.