I want to know how to upload a file in Java 8 using Spark framework 2.6.0 on Tomcat server version 8.5.9. I found an example but only applies to the standard configuration of Spark (embedded jetty). http://sparkjava.com/documentation#examples-and-faq
I solved this issue using Apache Commons Fileupload:
File archivo = new File("MyPath");
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setRepository(archivo);
ServletFileUpload fileUpload = new ServletFileUpload(factory);
List<FileItem> items = fileUpload.parseRequest(request.raw());
FileItem item = items.stream()
.filter(e -> "file".equals(e.getFieldName()))
.findFirst().get();
String fileName = item.getName();
item.write(new File(archivo, fileName));
Related
I am new to SolrJ. I have a requirement to index zip, pdf and html documents using the SolrJ Java API. Can anyone please give me some examples to index different types of documents, using SolrJ in java applications?
Is there any good links that i can go through where I can find good examples in Java to index different types of documents available in a folder...
Thank you for your help..
According to the output its clear that solrj is not indexing the .xml file that i am trying to can anyone pls comment what i am doing wrong...
Code:
String urlString = "http://localhost:8983/solr/tests";
HttpSolrClient solr = new HttpSolrClient.Builder(urlString).build();
solr.setParser(new XMLResponseParser());
File file = new File("D:/work/devtools/Solr/solr-7.6.0/example/exampledocs/hd.xml");
InputStream fis = new FileInputStream(file);
/* Tika specific */
ContentHandler contenthandler = new BodyContentHandler(10 * 1024 * 1024);
Metadata metadata = new Metadata();
metadata.set(Metadata.RESOURCE_NAME_KEY, "hd.xml");
ParseContext parseContext = new ParseContext();
// Automatically detect best parser base on detected document type
AutoDetectParser autodetectParser = new AutoDetectParser();
// OOXMLParser parser = new OOXMLParser();
autodetectParser.parse(fis, contenthandler, metadata, parseContext);
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", file.getCanonicalPath());
SolrQuery query = new SolrQuery("*.*");
// query.set("q", "price:599.99");
QueryResponse response = solr.query(query);
Output :
solr query{responseHeader={status=0,QTime=0,params={q=*.*,wt=xml,version=2.2}},response={numFound=0,start=0,docs=[]}}
links for basic information:https://www.youtube.com/watch?v=rxoS1p1TaFY&t=198s
2) https://lucene.apache.org/solr/ link to download the latest version
How to use solrj in Java Application:
java version should be 1.8
#)download solr latest version unzip
1)add the dependencies in your pom.xml file
org.apache.solr
solr-solrj
7.6.0
** start the solr from solr /bin folder and check the solr admin console by hitting this http://localhost:8983/solr/#
2)
basic example code:(this code is good enough to understand the solrj)
create the indexfiles core in solr and use the following code
String urlString = "http://localhost:8983/solr/indexfiles";
HttpSolrClient solr = new HttpSolrClient.Builder(urlString).build();
solr.setParser(new XMLResponseParser());
File file = new File("D:/work/devtools/Solr/solr-7.6.0/example/exampledocs/176444.zip");
ContentStreamUpdateRequest req = new ContentStreamUpdateRequest("/update/extract");
// req.addFile(file, "application/pdf");//change the content type for different input files
req.addFile(file, "text/plain");
String fileName = file.getName();
req.setParam("literal.id", fileName);
req.setAction(req.getAction().COMMIT, true, true);
NamedList<Object> result = solr.request(req);
int status = (Integer) ((org.apache.solr.common.util.SimpleOrderedMap) (result.get("responseHeader"))).get("status");
System.out.println("Result: " +result);
System.out.println("solr query"+ solr.query(new SolrQuery("*.*")));
3)query from the solr admin console using this http://localhost:8983/solr/indexfiles/select?q=SOLR1000
just change the text(q="<text to search>") that u want to search that available in the files that u indexed
u can find query parameter q in the solr admin console where we can give the required text to search if u are not comfortable with solr querys by default it is *:*
NOTE:dont need to think about Apache Tika and all to integrate it with Apache solr to index zip files and all because its by default available in solr new version
****Note: dont confuse by looking into the outputs from standalone admin(which gives complete data in the output ex: hd.xml is indexed which is available in the /exampledocs folder in solr) and the output u get by indexing the same files using solrj through java application
ex:solrj it will just index the file which means from the solr admin console u can see the following as out put when u fire query
(http://localhost:8983/solr/indexfiles/select?q=*:*)
output:
{
"id":"hd.xml",
"stream_size":["null"],
"x_parsed_by":["org.apache.tika.parser.DefaultParser",
"org.apache.tika.parser.xml.DcXMLParser"],
"stream_content_type":["text/xml"],
"content_type":["application/xml"],
"_version_":1624155471570010112},
But if we index throw command prompt using ---> java -Dc=name -jar post.jar *.xml the output contains the data available inside the xml file (http://localhost:8983/solr/indexfiles/select?q=*:*)
Xml Specific version of code to index xml file into Solr. But Xml should in below format.
<add>
<doc>
<field name="id">PMID</field>
<field name="year_i">Year</field>
<field name="name">ArticleTitle</field>
<field name="abstract_s">AbstractText</field>
<field name="cat">MeshHeading1</field>
<field name="cat">MeshHeading2</field>
</doc>
</add>
Below is the code to index xml data to Solr.
File xmlFile = new File("example.xml");
Reader fileReader = new FileReader(xmlFile);
BufferedReader bufReader = new BufferedReader(fileReader);
StringBuilder sb = new StringBuilder();
String line = bufReader.readLine();
while( line != null){
sb.append(line).append("\n");
line = bufReader.readLine();
}
String xml2String = sb.toString();
String urlString = String.format("http://localhost:8983/solr/%s", "pubmed1");
HttpSolrClient server = new HttpSolrClient.Builder(urlString).build();
server.setParser(new XMLResponseParser());
DirectXmlRequest xmlreq = new DirectXmlRequest( "/update", xml2String );
server.request( xmlreq );
server.commit();
Talking about Apache Tika it will help you out to extract the file content. The file can be xlsx,pdf,html,xml. In case of xml file format you need to write parser to convert xml format in solr xml format. You can use XSLT in case of xml.
refer in case of Apache Tika:-
enter link description here
I'm trying to programmatically upload a file to an enpoint via RestEasyClient.
File file = new File("/Users/michele/path/file.txt");
MultipartOutput multipartOutput = new MultipartOutput();
multipartOutput.addPart(file, MediaType.APPLICATION_OCTET_STREAM_TYPE, "file.txt");
Entity<MultipartOutput> entity = Entity.entity(multipartOutput, MediaType.MULTIPART_FORM_DATA_TYPE);
//client is an instance of org.jboss.resteasy.client.jaxrs.ResteasyClient
client
.target("http://localhost:8080/endpoint")
.request()
.post(entity);
The problem is that the backend does not "find" the file that I uploaded
backend code
DiskFileItemFactory factory = new DiskFileItemFactory();
ServletFileUpload fileUpload = new ServletFileUpload(factory);
List<FileItem> items = fileUpload.parseRequest(httpReq);
items is always empt.
Using MultipartFormDataOutput::addFormData, as described in many articles, works but does not fit my use case.
Also using apache.http.client.HttpClient works, but I prefer to avoid adding dependencies to my client.
Any ideas?
Found it.
The trick was to use MultipartFormDataOutput and to set the filename when adding a part
MultipartFormDataOutput multipartOutput = new MultipartFormDataOutput();
multipartOutput.addFormData("uploaded file", file, MediaType.APPLICATION_OCTET_STREAM_TYPE, "file.txt");
I'm using a servlet to do a multiple file upload (using Apache Commons FileUpload). A portion of my code is posted below. My problem is that if I upload files again and again , the memory consumption of the app server jumps rather drastically. The Apache Tomcat server seems to hang on to the memory and never return it. The heap space runs out of memory. Sometimes it runs out of memory exception and throws java heap space error.
I closed all the input streams, I think the problem is in the ServletFileUpload, could anyone help me out to how to close it.
ServletContext context=this.getServletConfig().getServletContext();
DiskFileItemFactory factory = new DiskFileItemFactory();
FileCleaningTracker fileCleaningTracker = FileCleanerCleanup.getFileCleaningTracker(context);
factory.setFileCleaningTracker(fileCleaningTracker);
if (isMultiPart) {
upload = new ServletFileUpload(factory);
try {
itr = upload.getItemIterator(request);
while (itr.hasNext()) {
item = itr.next();
if (item.isFormField()) {
...
You're using FileCleaningTracker, there are versions of Apache commons FileUpload with a bug in that component (see this: http://blog.novoj.net/2012/09/19/commons-file-upload-contains-a-severe-memory-leak/)
It seems it has been already fixed: https://issues.apache.org/jira/browse/FILEUPLOAD-189
So try using the last available version.
So I am using resumable.js to upload files to a server.
The directory that I want to save to is something like
/dir/files/upload/
Obviously just made up, but this directory has user permissions to write to it.
I am using JSP to listen to the POST request that resumable.js makes, and writing the
.part
files to that directory.
Sample listener:
<% if(request.getMethod().equals("POST") && request.getParameter("resumableFilename") != null){
long chunkSize = StringUtils.isEmpty(request.getParameter("resumableChunkSize"))? 0:Long.parseLong(request.getParameter("resumableChunkSize"));
String fileName = request.getParameter("resumableFilename");
long totalSize = StringUtils.isEmpty(request.getParameter("resumableTotalSize"))? 0:Long.parseLong(request.getParameter("resumableTotalSize"));
String temp_dir = "/dir/files/upload/"+request.getParameter("resumableIdentifier");//Add in user_id
String dest_dir = temp_dir+fileName+".part"+request.getParameter("resumableChunkNumber");
File fDir = new File(temp_dir);
fDir.mkdirs();
if(ServletFileUpload.isMultipartContent(request)){
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setRepository(new File(temp_dir)); ServletFileUpload upload = new ServletFileUpload(factory);
List items = upload.parseRequest(request);
ArrayListIterator iter = (ArrayListIterator)items.iterator();
FileItem item = (FileItem)iter.next();
File fileWithNewDir = new File(dest_dir);
item.write(fileWithNewDir); // write file to dest_dir (fileName.part*CHUNK_NUM*)
}
}
%>
The script is hosted on
www.site.com/pubs/res.jsp
According to the JS itself for resumable, the process of uploading it gets completed, however, a new directory is not made at all. I know it's not the write permissions, so it must be something else.
Here is my call in javascript for a new resumable object
var resume = new Resumable({
target:"res.jsp",
resumableChunkSize: 1*1024*1024,
simultaneousUploads: 3,
testChunks: false,
throttleProgressCallbacks: 1
});
It seems to be hitting the jsp file, but nothing is happening.
I followed Apache's fileupload page in order to implement that listener, but maybe I went wrong at some point.
Apache's FileUpload
Resumable.js
Location of the directory matters. It has to be within the context of the WAR. You cannot write to any location outside the context of the container. If you look at the log you may be abe to see the error message which can explain this.
I want to upload files outisde web server like in d d drive into servlets, I but I'm not able to upload them.
What I have to do to make functionality like this enable in Tomcat 6.0?
This ought just to work. All you basically need to do is to obtain the uploaded file in flavor of an InputStream from the request body. You normally use Apache Commons FileUpload for this. Then you can write it to any OutputStream you like the usual Java IO way, such as FileOutputStream.
Assuming that you're actually using Apache Commons FileUpload which requires Apache Commons IO as a dependency, here's a basic example:
String filename = FilenameUtils.getName(fileItem.getName()); // Important!
File destination = new File("D:/path/to/files", filename);
InputStream input = null;
OutputStream output = null;
try {
input = fileItem.getInputStream();
output = new FileOutputStream(destination);
IOUtils.copy(input, output);
} finally {
IOUtils.closeQuietly(output);
IOUtils.closeQuietly(input);
}
Alternatively you can also just use the Fileupload's convenienced FileItem#write() method:
String filename = FilenameUtils.getName(fileItem.getName()); // Important!
File destination = new File("D:/path/to/files", filename);
fileItem.write(destination);
For more examples, hints and tricks, check the FileUpload User Guide and FAQ.