In my software I want to store some data, that later they will be used. Something like a database to hold data:
Date, source path, destination path, and an array of file names.
Also another table to hold information about ftp connection:
Host, port, username and password
I need to know what methods are available to store and parse these data. I noticed there is a file type called .csv, is this an option for me? And is there any other option?
I think this depends a lot on how much data you want to store and how you need to access it.
If your application is going to be collecting a lot of structured data, such as user profiles, or product information, ie, if your application is all about a database then, yes as others have commented some sort of SQL database would make sense.
If your needs are more along the lines of just storing some "session" information, maybe like the last state of a GUI form for example, you might want to just serialize the data and write it to a simple text file.
One simple way to do that would be to serialize the data in a human readable format such as JSON and then write the text to a file, and then read it back and deserialize it when you need to restore it from storage.
If this is what you are looking for take a look at gson (from google), it provides a very easy what to convert a java object to JSON and back again.
JSON, is just text, so you can just read and write it to a simple text file.
Related
I'm using Java's email class to send an email through our SMTP server. I want to add a feature where if a user resubmits their form, I email them an attachment with a copy of the old data and their new data they are submitting, so they can see the changes they are making. I don't have a problem with this flow, it's when I want to add them to my email I run into issues. Most of the solutions I've seen for adding attachments to a java email consist of attaching a system file, but these objects are generated at runtime, and I don't want to start adding files to the system just for this feature, so I wanted to know if I can add objects specifically as attachments.
If you were going to output the object to the console for debugging purposes, how would you do that? Would you .ToString() it, would you iterate over the properties and output them as individual strings?
Rather than think about the object as an object, think about how you want to show it to your users. If this involves a lot of rows with descriptors so they know what the data mean, then output it like that. If they are developers and expect to see it as JSON, output it formatted like that.
Most likely, you'll have to iterate over the object in some way to output it as strings in the body itself.
If you really want to add an attachment, you can create a temporary file (with a random name to prevent overwriting the file), send the email with the attachment, then delete the file.
I was going to say that there's probably a way to create a file in memory without saving it to disk that you can use as an attachment, but maybe there isn't in Java.
Create a File object in memory from a string in Java
I need some suggestions for the App am developing,
I have some 6 sensors which are continuously sending the data to the App, I need to log the data in some file. I want to know which is the better approach to store such data. I tired using json by storing all the values in one json objects, and created a array of json objects somewhat like this
[{sensor1, senor2, sensor3, sensor4, sensor5, sensor6},{sensor1, senor2, sensor3, sensor4, sensor5, sensor6}]
But I have more then 20000 data like this to store, creating the array of 20000 json object and writing to file in one shot looks expensive. on that data is coming once in 15 seconds. writing one JSON object once in every 15 second will disturb the JSON format.
So I need suggestion like is using JSON method to store this data is better or I should think of some other method like CSV??
If you want to store the data points over time, have you considered using a time series database? There are various NoSQL style databases that are well suited to this task, Cassandra being one
https://academy.datastax.com/demos/getting-started-time-series-data-modeling
Without a doubt, CSV.
But only if you're certain your format will not change (always 6 sensors). Furthermore, in CSV, data can be added at the end of a file while in JSON you have to write in the middle.
We have seen lot of applications who are working with JSON file but i have a case study of which i want to get solution.
Let us see ...
a app is working with json file which gets requests from million users and every second thousands of requests has been completed.
JSON file is updated by admin panel every minute or second or specific time frame.
what will be behaviour of JSON file while request has been received to access JSON file and open for update from admin at same time (i have read it that JSON file will be fetched in readable mode.)
Let JSON file is writing using some script and its process is third of a second than what will be behaviour while 50% file has been updated.
Either file will be given with new written content when process completed or when it was partially updated?
Don't bother with locking, just use rename().
Assuming you're running on an OS where a rename() is an atomic operation, create a new file, say "/data/file/name.json.new", then when that's complete, rename the file. In C that would look like this:
rename( "/data/file/name.json.new", "/data/file/name.json" );
This way, any process opening "/data/file/name.json" will always see a consistent data file.
Practically, by what you describe, you want a service that applies operations on a file server-side.
You should though avoid taking the responsibility of Creating, Readind, Updating and Deleting (CRUD), as you will have troubles on preserving principles such as Atomicity, Consistency, Isolation and Durability (ACID), while there are systems doing that for you, the Database Management Systems.
In simple words, scenarios like what you describe should be a responsibility of a DBMS and not yours.
You probably need a NoSQL DBMS, that responsible for the CRUD operations of your database - which can be file-based in a JSON format and other forms, preserving ACID always (or almost always, but this is probably something you will learn on searching on it). MongoDB is a great example of such system.
Because you mentioned JSON, please take into consideration that it is another story to transfer the data, and another to store them. I suggest that you use the JSON format for requests & responses, but explore other options in storage. For instance, even a Relational DBMS that uses SQL can be good for you, it always depends on your needs. You might just need to form (encode & decode) the data in JSON format wherever received or sent to each client.
Take a look here for more info.
I am making an auto chat client like Cleverbot for school. I have everything working, but I need a way to make a knowledge base of responses. I was going to make a matrix with all the responses that I need the bot to say, but I think it would be hard to edit the code every time I want to add a responses to the bot. This is the code that I have for the knowledge base matrix:
`String[][] Database={
{"hi","hello","howdy","hey"},//possible user input
{"hi","hello","hey"},//response
{"how are you", "how r u", "how r you", "how are u"},
{"good","doing well"}`
How would I make a matrix like this from a text file? Is there a better way than reading from a text file to deal with this?
You could...
Use a properties file
The properties file is something that can easily be read into (and stored from, but you're not interested in that) Java. The class java.util.Properties can make that easier, but it's fairly simple to load it and then you access it like a Map.
hello.input=hi,hello,howdy,hey
hello.output=hi,hello,hey
Note the matching formats there. This has its own set of problems and challenges to work with, but it lets you easily pull things in to and out of properties files.
Store it in JSON
Lots of things use JSON for a serialization format. And thus, there are lots of libraries that you can use to read and store from it. It would again make some things easier and have its own set of challenges.
{
"greeting":{
"input":["hi","hello","howdy","hey"],
"output":["hi","hello","hey"]
}
}
Something like that. And then again, you read this and store it into your data structures. You could store JSON in a number of places such as document databases (like couch) which would make for easy updates, changes, and access... given you're running that database.
Which brings us to...
Embedded databases
There are lots of databases that you can stick right in your application and access it like a database. Nice queries, proper relationships between objects. There are lots of advantages to using a database when you actually want a database rather than hobbling strings together and doing all the work yourself.
Custom serialization
You could create a class (instead of a 2d array) and then store the data in a class (in which it might be a 2d array, but that's an implementation detail). At this point, you could implement Serializable and write the writeObject and readObject methods and store the data somehow in a file which you could then read back into the object directly. If you have the administration ability of adding new things as part of this application (or another that uses the same class) you could forgo the human readable aspect of it and use the admin tool (that you write) to update the object.
Lots of others
This is just the tip of the iceberg. There are lots of ways to go about this.
P.S.
Please change the name of the variable from Database to something in lower case that better describes it such as input2output or the like. Upper case names are typically reserved for class names (unless its all upper case, in which case it's a final static field)
A common solution would be to dump the data in to a properties file, and then load it with the standard Properties.load(...) method.
Once you have your data like that, you can then access the data by a map-like interface.
You could find different ways of storing the data in the file like:
userinput=hi,hello,howdy,hey
response=hi,hello,hey
...
Then, when you read the file, you can split the values on the comma:
String[] expectHello = properties.getProperty("userinput").split(",");
Writing Java objects or a List into a text file is ok. But I want to know how I can update or rewrite a object which was written previously without writing objects again. For example, let s assume there is a java.util.List has a set of Objects. and then that list is written to a text file. Then later that file will be read again and get all objects from list and then change one object's value at run time by a java application. Then I don't need to write entire list back to the text file. Instead only the updated object in the list is required to be rewritten or updated in the text file without rewriting the whole list again. Any suggestion, or helpful source with sample codes please.
Take a look at RandomAccessFile. This will let you seek to the place in the file you want, and only update the part that you want to update.
Also take a look at this question on stackoverflow.
Without some fairly complex logic, you won't usually be able to update an object without rewriting the entire file. For example, if one of the objects on your list contains a string "shortstring", and you need to update it with string "muchmuchlongerstring", there will be no space in the file for the longer string without rewriting all the following content in the file.
If you want to persist large object trees to a file and still have the ability to update them, your code will be less buggy and life will be simplified by using one of the many file-based DBs out there, like:
SQLite (see Java and SQLite)
Derby
H2 (disk-based tables)