I'm trying to write a parser and and I'm using MongoDB as a database. Essentially it is going through, creating objects, and then saving them. It is doing this several times a second. After around 164 objects are saved it crashes with this error:
com.mongodb.MongoException: isOk() at
com.mongodb.DBTCPConnector._checkWriteError(DBTCPConnector.java:130)
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:142) at
com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:141) at
com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:97) at
com.mongodb.DBCollection.insert(DBCollection.java:61) at
com.mongodb.DBCollection.save(DBCollection.java:547) at
com.google.code.morphia.DatastoreImpl.save(DatastoreImpl.java:638) at
com.google.code.morphia.DatastoreImpl.save(DatastoreImpl.java:685) at
com.google.code.morphia.DatastoreImpl.save(DatastoreImpl.java:679) at
com.soleo.internal.releasenotes.orm.Storage.save(Storage.java:764) at
com.soleo.internal.releasenotes.page.MainPage$2.onSubmit(MainPage.java:256)
Now at one point I had over 1000 objects in this same database, I just didn't insert them all at once. So it can't be a hard drive space issue. I can't find any documentation at all of this error online. Oddly it's only when I try saving THIS object. If I try saving Object B after the crash it saves just fine. It just crashes on Object A, the one I initially mass saved.
I ran a test multiple times and it failed in the same place. I used random values so I could prove it wasn't a variable issue:
FIRST TRY:
==============================
CREATING RELEASE #162
Component: iHateYou
Location: 250344
Version: 8.8.1.5-2
Date: Sun Feb 07 00:00:00 EST 3188 (02-07-3188)
SAVING.............
SUCCESS.
==============================
CREATING RELEASE #163
Component: iHateYou
Location: 227407
Version: 5.5.7.6-7
Date: Sat Mar 04 00:00:00 EST 439 (03-04-439)
SAVING.............
SUCCESS.
==============================
CREATING RELEASE #164
Component: iHateYou
Location: 38694
Version: 3.5.4.7-7
Date: Mon Jan 03 00:00:00 EST 158 (01-03-158)
SAVING.............
Oct 28, 2011 11:17:11 AM org.apache.wicket.RequestCycle logRuntimeException
SEVERE: isOk()
com.mongodb.MongoException: isOk()
at com.mongodb.DBTCPConnector._checkWriteError(DBTCPConnector.java:130)
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:142)
SECOND TRY:
==============================
CREATING RELEASE #162
Component: iHateYou
Location: 64717
Version: 0.1.0.4-8
Date: Sun May 07 00:00:00 EST 971 (05-07-971)
SAVING.............
SUCCESS.
==============================
CREATING RELEASE #163
Component: iHateYou
Location: 19360
Version: 4.5.8.1-3
Date: Wed Aug 04 00:00:00 EST 1339 (08-04-1339)
SAVING.............
SUCCESS.
==============================
CREATING RELEASE #164
Component: iHateYou
Location: 115518
Version: 0.0.8.0-2
Date: Sat Apr 07 00:00:00 EST 143 (04-07-143)
SAVING.............
Oct 28, 2011 11:15:28 AM org.apache.wicket.RequestCycle logRuntimeException
SEVERE: isOk()
com.mongodb.MongoException: isOk()
at com.mongodb.DBTCPConnector._checkWriteError(DBTCPConnector.java:130)
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:142)
Here's some partially obfuscated code:
Random blank = new Random();
ObjectRef blah = new ObjectRef("iHateYou");
storage.save(blah);
for(int i = 0; i < 300; i++)
{
System.out.println("==============================\nCREATING OBJECT #" + i);
ObjectA saveMe = new ObjectA();
saveMe.setRef(storage.getRefByName("iHateYou"));
System.out.println("Component: " + saveMe.getRef.getName());
saveMe.setLocation(blank.nextInt(300000) + "");
System.out.println("Location: " + saveMe.getLocation());
saveMe.setVersion(new Version(blank.nextInt(9) + "." + blank.nextInt(9) + "." + blank.nextInt(9) + "." + blank.nextInt(9) + "-" + blank.nextInt(9)));
System.out.println("Version: " + saveMe.getVersion());
try
{
String randomDate = "0" + blank.nextInt(9) + "-0" + blank.nextInt(9) + "-" + blank.nextInt(4000);
saveMe.setReleaseDate(new SimpleDateFormat("MM-dd-yyyy").parse(randomDate));
System.out.println("Date: " + saveMe.getReleaseDate() + " (" + randomDate + ") ");
}
catch (ParseException e)
{
e.printStackTrace();
}
System.out.println("SAVING.............");
storage.save(saveMe);
System.out.println("SUCCESS.");
}
Sounds like it could be a variation of this bug: https://jira.mongodb.org/browse/RUBY-324
Are you using sharding? In that case your config DB might be corrupt. The driver is probably not expecting to receive the message "isOk" back.
Please tell us more about your environment: MongoDB version, using sharding or not, driver version, etc.
Related
In my jHipster project I want to search row by datetime range.
The fragments of my code are:
product-inquiry-view.component.html
<div class="input-group date" id='datetimepicker'>
<input id="field_dateStart" #dateStart="ngbDatepicker" [(ngModel)]="criteriaByStartDate"/>
<input id="field_dateEnd" #dateEnd="ngbDatepicker" [(ngModel)]="criteriaByEndDate"/>
<button (click)="findByCriteria()"
</button>
</div>
Component class:
product-inquiry-view.component.ts
export class ProductInquiryViewComponent implements OnInit, OnDestroy {
criteriaByStartDate: string;
criteriaByEndDate: string;
..........
findByCriteria() {
criteriaByStartDate: Date;
criteriaByEndDate: Date;
let criteriaByStartDate = this.criteriaByStartDate;
let criteriaByEndDate = this.criteriaByEndDate
this.productInquiryViewService
.query({
'piDate.greaterThan': criteriaByStartDate,
'piDate.lessThan': criteriaByEndDate,
'piDocnum.contains': this.criteriaByDocnum
}).subscribe(
(res: HttpResponse<IProductInquiryView[]>) => {
this.productInquiryViews = res.body;
},
(res: HttpErrorResponse) => this.onError(res.message)
);
}
Controller:
ProductInquiryViewResource.java
#GetMapping("/product-inquiry-views")
#Timed
public ResponseEntity<List<ProductInquiryViewDTO>> getAllProductInquiryViews(ProductInquiryViewCriteria criteria) {
log.debug("REST request to get ProductInquiryViews by criteria: {}", criteria);
List<ProductInquiryViewDTO> entityList = productInquiryViewQueryService.findByCriteria(criteria);
return ResponseEntity.ok().body(entityList);
}
And I got exception:
2019-04-15 12:48:45.910 WARN 24330 --- [ XNIO-2 task-1] .m.m.a.ExceptionHandlerExceptionResolver : Resolved exception caused by handler execution: org.springframework.validation.BindException:
org.springframework.validation.BeanPropertyBindingResult: 2 errors
Field error in object 'productInquiryViewCriteria' on field 'piDate.greaterThan': rejected value [Mon Apr 01 2019 00:00:00 GMT 0300]; codes
[typeMismatch.productInquiryViewCriteria.piDate.greaterThan,typeMismatch.piDate.greaterThan,typeMismatch.greaterThan,typeMismatch.java.time.ZonedDateTime,typeMismatch]; arguments
[org.springframework.context.support.DefaultMessageSourceResolvable: codes [productInquiryViewCriteria.piDate.greaterThan,piDate.greaterThan]; arguments [];
default message [piDate.greaterThan]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.time.ZonedDateTime'
for property 'piDate.greaterThan'; nested exception is org.springframework.core.convert.ConversionFailedException: Failed to convert from type [java.lang.String]
to type [java.time.ZonedDateTime] for value 'Mon Apr 01 2019 00:00:00 GMT 0300'; nested exception is java.lang.IllegalArgumentException: Parse attempt failed for value [Mon Apr 01 2019 00:00:00 GMT 0300]] enter code here
I see that "Mon Apr 01 2019 00:00:00 GMT 0300" is not correst string for parsing to ZoneDateTime, so I tried send criteria in such format :
"2019-04-01T00:15:30+03:00[Europe/Moscow]";
Same result.
Should be :
this.productInquiryViewService
.query({
'piDate.greaterThan': criteriaByStartDate.toISOString(),
'piDate.lessThan': criteriaByEndDate.toISOString(),
...
I execute an AWS command to retrieve the spot price history.
DescribeSpotPriceHistoryRequest request = new DescribeSpotPriceHistoryRequest().withEndTime(start)
.withInstanceTypes("m1.xlarge").withProductDescriptions("Linux/UNIX (Amazon VPC)").withStartTime(end);
DescribeSpotPriceHistoryResult response = client.describeSpotPriceHistory(request);
System.out.println(response.toString());
I obtained the result in json format but I receive it in String like:
{SpotPriceHistory: [{AvailabilityZone: us-east-1d,InstanceType: m1.xlarge,ProductDescription: Linux/UNIX,SpotPrice: 0.035000,Timestamp: Wed Nov 07 00:18:50 CET 2018}, {AvailabilityZone: us-east-1c,InstanceType: m1.xlarge,ProductDescription: Linux/UNIX,SpotPrice: 0.035000,Timestamp: Wed Nov 07 00:18:50 CET 2018}, {AvailabilityZone: us-east-1b,InstanceType: m1.xlarge,ProductDescription: Linux/UNIX,SpotPrice: 0.035000,Timestamp: Wed Nov 07 00:18:50 CET 2018}, {AvailabilityZone: us-east-1a,InstanceType: m1.xlarge,ProductDescription: Linux/UNIX,SpotPrice: 0.035000,Timestamp: Wed Nov 07 00:18:50 CET 2018}, {AvailabilityZone: us-east-1e,InstanceType: m1.xlarge,ProductDescription: Linux/UNIX,SpotPrice: 0.350000,Timestamp: Thu Sep 20 01:08:39 CEST 2018}]}
my question is: How can I improve the displaying of the results ? like
{
"Timestamp": "2018-09-08T17:07:14.000Z",
"AvailabilityZone": "us-east-1d",
"InstanceType": "p2.16xlarge",
"ProductDescription": "Linux/UNIX",
"SpotPrice": "4.320000"
},
{
"Timestamp": "2018-09-08T17:07:14.000Z",
"AvailabilityZone": "us-east-1c",
"InstanceType": "p2.16xlarge",
"ProductDescription": "Linux/UNIX",
"SpotPrice": "4.320000"
},
{
"Timestamp": "2018-09-08T17:07:14.000Z",
"AvailabilityZone": "us-east-1b",
"InstanceType": "p2.16xlarge",
"ProductDescription": "Linux/UNIX",
"SpotPrice": "4.320000"
},
{
"Timestamp": "2018-09-08T12:32:28.000Z",
"AvailabilityZone": "us-east-1e",
"InstanceType": "p2.16xlarge",
"ProductDescription": "Linux/UNIX",
"SpotPrice": "4.320000"
}
]
}
You're calling the .toString() on the response object, just be careful with this as there is no guarantee that will always be json, as you're seeing above, it's not even valid json as it's missing quotes around the attribute names and values.
One option to get what you want is to call response.getSpotPriceHistory() to get you the array of spot prices, then pass that thru ObjectMapper and write it as a pretty string, like so:
public static void main(String[] args) throws IOException {
AmazonEC2 client = AmazonEC2Client.builder().build();
DescribeSpotPriceHistoryRequest request = new DescribeSpotPriceHistoryRequest()
.withEndTime(new Date())
.withInstanceTypes("m1.xlarge").withProductDescriptions("Linux/UNIX (Amazon VPC)")
.withStartTime(new Date());
DescribeSpotPriceHistoryResult response = client.describeSpotPriceHistory(request);
ObjectMapper mapper = new ObjectMapper();
String asPrettyJSon = mapper.writerWithDefaultPrettyPrinter()
.writeValueAsString(response.getSpotPriceHistory());
System.out.println(asPrettyJSon);
}
Both represent a json array containing jsonObjects of same structure.
Displaying result will depend on your front implementation not the layaout of your jsonRespense.
I want to delete the last newline present in my file using java. I mean, there is a newline at the very end of the file, which I want to remove.
I have tried many solutions provided online, but nothing works.
The below code removes all newlines from file
trimm.replace("\n", "").replace("\r", "");
Sample text:
ABC 123|1|2 ABC '123|1|2|"Jan 30 2018 2:34:13:000AM"|dd1|1|"Jan 30 2018 2:56:08:000AM"|EST' ABC 20180821
ABC 123|1|2 ABC '123|1|2|"Jan 30 2018 2:34:13:000AM"|dd1|1|"Jan 30 2018 2:56:08:000AM"|EST' ABC 20180821
The above sample has newline at the end. I have referred the URLs below:
http://www.avajava.com/tutorials/lessons/how-do-i-remove-a-newline-from-the-end-of-a-string.html
https://www.java-forums.org/new-java/22655-removing-last-blank-line-txt-file.html
I can't use split() after \n as many raws have same word
My code:
String actual ="ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\r\n" +
"ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\r\n";
try {
File fout = new File("I:\\demo\\S2.txt");
FileOutputStream fos = new FileOutputStream(fout);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(fos));
String trimm= actual;
/* StringBuilder sb = new StringBuilder(trimm);
int lastEnterPosition = trimm.lastIndexOf("\r\n");
sb.replace(lastEnterPosition, lastEnterPosition + 1, "");
trimm = sb.toString();*/
trimm = trimm.replaceAll("[\n\r]+$", "");
bw.write(trimm);
bw.newLine();
bw.close();
} catch (FileNotFoundException e){
// File was not found
e.printStackTrace();
} catch (IOException e) {
// Problem when writing to the file
e.printStackTrace();
}
Any workaround will be helpful.
You can use replaceFirst which use regex with this one [\n\r]+$, like so :
trimm = trimm.replaceFirst("[\n\r]+$", "");
Full code
I tried this piece of code :
public static void main(String[] args) throws Exception {
String trimm = "ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\n" +
"ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\r\n";
System.out.println("---------------------------------------------------Before replace Start of the input---------------------------------------------------");
System.out.println(trimm);
System.out.println("---------------------------------------------------Before replace End of the input---------------------------------------------------");
System.out.println("---------------------------------------------------After replace Start of the input---------------------------------------------------");
trimm = trimm.replaceFirst("[\n\r]+$", "");
System.out.println(trimm);
System.out.println("---------------------------------------------------After replace End of the input---------------------------------------------------");
}
The output :
---------------------------------------------------Before replace Start of the input---------------------------------------------------
ABC 123|1|2 ABC '123|1|2|"Jan 30 2018 2:34:13:000AM"|dd1|1|"Jan 30 2018 2:56:08:000AM"|EST' ABC 20180821
ABC 123|1|2 ABC '123|1|2|"Jan 30 2018 2:34:13:000AM"|dd1|1|"Jan 30 2018 2:56:08:000AM"|EST' ABC 20180821
---------------------------------------------------Before replace End of the input---------------------------------------------------
---------------------------------------------------After replace Start of the input---------------------------------------------------
ABC 123|1|2 ABC '123|1|2|"Jan 30 2018 2:34:13:000AM"|dd1|1|"Jan 30 2018 2:56:08:000AM"|EST' ABC 20180821
ABC 123|1|2 ABC '123|1|2|"Jan 30 2018 2:34:13:000AM"|dd1|1|"Jan 30 2018 2:56:08:000AM"|EST' ABC 20180821
---------------------------------------------------After replace End of the input---------------------------------------------------
Note that there are a break line before the replace and after the replace only the last break line is removed.
Ideone demo
More details
I tried this three solutions :
Code 1 (Your code)
public static void main(String[] args) throws Exception {
String trimm = "ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\n" +
"ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\r\n";
try {
File fout = new File("path");
FileOutputStream fos = new FileOutputStream(fout);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(fos));
trimm = trimm.replaceAll("[\n\r]+$", "");
bw.write(trimm);
//bw.newLine();//<-----------------------note this
bw.close();
} catch (FileNotFoundException e) {
// File was not found
e.printStackTrace();
} catch (IOException e) {
// Problem when writing to the file
e.printStackTrace();
}
}
Code 2
public static void main(String[] args) throws Exception {
String trimm = "ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\n" +
"ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\r\n";
Path path = Paths.get("path");
try (BufferedWriter writer = Files.newBufferedWriter(path))
{
writer.write(trimm.replaceFirst("[\n\r]+$", ""));
}
}
Code 3
public static void main(String[] args) {
String trimm = "ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\n" +
"ABC 123|1|2 ABC '123|1|2|\"Jan 30 2018 2:34:13:000AM\"|dd1|1|\"Jan 30 2018 2:56:08:000AM\"|EST' ABC 20180821\r\n";
try {
Files.write(Paths.get("path"), trimm.replaceFirst("[\n\r]+$", "").getBytes());
} catch (IOException e) {
e.printStackTrace();
}
}
and all the three codes gives me :
If you know that the string will always end with \r\n, given
String test = "your string that ends with a newline\r\n";
You can use something like
String eol = "\r\n";
int eolLen = eol.length();
String tmp = test.substring(0, test.length()-eolLen);
Using a regex seems a bit overkill. You could optionally check to see that the string really ends with a newline and throw an exception of some kind or another if it's possible that you get bad input. Something like:
String check = test.substring(test.length() - eolLen);
if (!eol.equals(check)) {
throw new Exception(String.format("Expected newline, found %s", check));
}
This question already has answers here:
MongoDB extracting values from BasicDBObject (Java)
(2 answers)
Closed 5 years ago.
I want to get an JSON array into a var in java. My JSON seems ok but when i try to put a JSOn array into a java array var it does not work.
here the error i get : errororg.json.JSONException: JSONObject["tweets"] is not a JSONArray.
MongoClient mongoClient = new MongoClient( "localhost" , 27017 );
DB db = mongoClient.getDB( "test1" );
DBCollection coll = db.getCollection("tweetsCol");
DBCursor cursor = coll.find();
while (cursor.hasNext()) {
BasicDBObject obj = (BasicDBObject) cursor.next();
JSONObject objjj = new JSONObject(obj);
try{
System.out.println("okok "+objjj); // THE JSON I WILL SHOW YOU
JSONArray jsonMainArr = objjj.getJSONArray("tweets");
}catch(JSONException e){
System.out.println("error"+e);
}}
Here's my data in MongoDB:
{
"_id":"5939bc6676abbe186feb73a5",
"user_request_id":"5941903f37aaa6ec55689e85",
"tweets":[
{
"date":"Wed Jun 07 18:32:57 CDT 2017",
"text":"[Earthview Wonders][Video] No.265: Astronaut Thomas Pesquet completed 6-month #MissionProxima. #Neweyes\u2026 ",
"_id":872597276891398144,
"user":"livearthjp"
},
{
"date":"Wed Jun 07 18:16:56 CDT 2017",
"text":"Astronaut Thomas Pesquet #Thom_astro Shares His #Songs4Space ",
"_id":872593245716467712,
"user":"anasia5mice"
},
{
"date":"Wed Jun 07 15:46:03 CDT 2017",
"text":"Thomas Pesquet: Undocking and landing ",
"_id":872555275387117570,
"user":"GRASSIFREE"
},
{
"date":"Wed Jun 21 17:02:37 CDT 2017",
"text":"#Thom_astro #Space_Station And his colleagues said, 'Pesquet, if you play Baker Street one more time...'",
"_id":877647972430823429,
"user":"kimkemmis"
},
{
"date":"Wed Jun 21 17:01:16 CDT 2017",
"text":"[News] ",
"_id":877647632524394497,
"user":"ArthurC2Pouce"
},
{
"date":"Wed Jun 21 11:28:48 CDT 2017",
"text":"Thomas Pesquet's music is OUT THERE! Cool dude. ",
"_id":877563967178104836,
"user":"tiarudd34"
},
{
"date":"Wed Jun 21 11:10:15 CDT 2017",
"text":"jaime thomas pesquet",
"_id":877559296741048320,
"user":"sosthene_maus"
},
{
"date":"Wed Jun 21 10:23:03 CDT 2017",
"text":"French astronaut Thomas Pesquet took some of the most amazing pictures ever while in spce ",
"_id":877547418606329861,
"user":"raygibbs1"
},
{
"date":"Wed Jun 21 10:23:00 CDT 2017",
"text":"French astronaut Thomas Pesquet shares stunning pictures of Earth: via #AOL",
"_id":877547405180157952,
"user":"raygibbs1"
},
{
"date":"Wed Jun 21 08:46:13 CDT 2017",
"text":"Coll Cambuston like Thomas Pesquet! #thomastro #cardierun",
"_id":877523048546676736,
"user":"CambF974"
},
{
"date":"Wed Jun 21 08:00:06 CDT 2017",
"text":"Thomas Pesquet returned to Earth on 2 June 2017 after completion of his six-months long Proxima mission to the... ",
"_id":877511443775619072,
"user":"rospaceagency"
},
{
"date":"Tue Jun 20 23:50:34 CDT 2017",
"text":"Thomas Pesquet #Thom_astro and Messier 83 #Astronauts #ESA #CNES #NASA ✨👨\u200d🚀✨ ",
"_id":877388248368033794,
"user":"AmirAliBehrooz"
}
],
"query_name":"Pesquet",
"active":"true",
"started_at":"2017_06_08"
}
Now I understand better your question thanks to #Neil Lunn, and following his best advice, you have to understand that you are not dealing here with Json, you can directly do:
MongoClient mongoClient = new MongoClient( "localhost" , 27017 );
DB db = mongoClient.getDB( "test1" );
DBCollection coll = db.getCollection("tweetsCol");
DBCursor cursor = coll.find();
while (cursor.hasNext()) {
BasicDBObject obj = (BasicDBObject) cursor.next();
try{
System.out.println("okok "+obj); // THE JSON I WILL SHOW YOU
BasicDBList jsonMainArr = obj.get("tweets");
}catch(JSONException e){
System.out.println("error"+e);
}}
I wanted to write code which could read a log file and split it into multiple events, using the timestamp as the splitter (since every log entry begins with a timestamp). A sample of the logs I want to split is given below. I'd also like to keep the timestamp itself.
So if this is my input:
01 Aug 2016 04:48:13,311 ERROR [pool-2-thread-12436] com.orders.queue.OrdersQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Exception while calling /purchases: x-company-status : UNKNOWN_ERROR response status: Internal Server Error
01 Aug 2016 04:48:13,311 WARN [pool-2-thread-12436] com.orders.queue.OrdersQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Failed to process order, will be re-tried: ADD2500051FR
01 Aug 2016 04:48:13,332 INFO [pool-2-thread-12436] com.delegate - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Getting Email from Primary email
01 Aug 2016 04:48:13,363 WARN [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Message processing failed QueueMessage [payload=ADD2500051FR, delaySeconds=0, sqsId=51f70e3f-554a-463b-8384-0b2c25a90450, stringAttributes={features=adac2911-0578-4bcd-b8c3-783481a48e1d, accept-language=FR_FR, request-id=836ac8b6-515d-4414-b4c6-ddd8a52ef497}]
com.orders.exception.orderserviceException: Error in Calling PUT purchase from main service
at com.OrderServiceDelegate.handleInternalServerErrors(OrderServiceDelegate.java:352)
at com.OrderServiceDelegate.sendOrderForProcessing_aroundBody0(OrderServiceDelegate.java:113)
at com.OrderServiceDelegate.sendOrderForProcessing_aroundBody1$advice(OrderServiceDelegate.java:37)
at com.OrderServiceDelegate.sendOrderForProcessing(OrderServiceDelegate.java:1)
at com.orders.queue.OrdersQueueWorker.doWork(OrdersQueueWorker.java:168)
at com.queue.SQSQueueWorker.lambda$0(SQSQueueWorker.java:149)
at com.queue.SQSQueueWorker.dt_access$492(SQSQueueWorker.java)
at com.queue.SQSQueueWorker$$dtt$$Lambda$8/852112146.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
01 Aug 2016 04:48:13,365 INFO [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Order will be re-tried after 300 seconds: ADD2500051FR
01 Aug 2016 04:48:15,600 INFO [myScheduler-3] com.queue.SQSQueueWorker - x-company-requestid=sqs-worker service_name=orders_v3 Processing messages message_number=1
01 Aug 2016 04:48:15,600 INFO [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Received msg from SQS:QueueMessage [payload=428CB476547214700268914651663, delaySeconds=0, sqsId=7f4dcbbe-90c4-4e56-b4ab-50332597b5d8, stringAttributes={features=FIS-JEM, accept-language=EN_US, request-id=a2c31da4-517f-40ec-8587-624f97393659}]
Then my output should be (the horizontal line depicts where one entry ends and another begins):
01 Aug 2016 04:48:13,311 ERROR [pool-2-thread-12436] com.orders.queue.OrdersQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Exception while calling /purchases: x-company-status : UNKNOWN_ERROR response status: Internal Server Error
//--------------------------------------------
01 Aug 2016 04:48:13,311 WARN [pool-2-thread-12436] com.orders.queue.OrdersQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Failed to process order, will be re-tried: ADD2500051FR
//--------------------------------------------
01 Aug 2016 04:48:13,332 INFO [pool-2-thread-12436] com.delegate - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Getting Email from Primary email
//--------------------------------------------
01 Aug 2016 04:48:13,363 WARN [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Message processing failed QueueMessage [payload=ADD2500051FR, delaySeconds=0, sqsId=51f70e3f-554a-463b-8384-0b2c25a90450, stringAttributes={features=adac2911-0578-4bcd-b8c3-783481a48e1d, accept-language=FR_FR, request-id=836ac8b6-515d-4414-b4c6-ddd8a52ef497}]
com.orders.exception.orderserviceException: Error in Calling PUT purchase from main service
at com.OrderServiceDelegate.handleInternalServerErrors(OrderServiceDelegate.java:352)
at com.OrderServiceDelegate.sendOrderForProcessing_aroundBody0(OrderServiceDelegate.java:113)
at com.OrderServiceDelegate.sendOrderForProcessing_aroundBody1$advice(OrderServiceDelegate.java:37)
at com.OrderServiceDelegate.sendOrderForProcessing(OrderServiceDelegate.java:1)
at com.orders.queue.OrdersQueueWorker.doWork(OrdersQueueWorker.java:168)
at com.queue.SQSQueueWorker.lambda$0(SQSQueueWorker.java:149)
at com.queue.SQSQueueWorker.dt_access$492(SQSQueueWorker.java)
at com.queue.SQSQueueWorker$$dtt$$Lambda$8/852112146.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
//--------------------------------------------
01 Aug 2016 04:48:13,365 INFO [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Order will be re-tried after 300 seconds: ADD2500051FR
//--------------------------------------------
01 Aug 2016 04:48:15,600 INFO [myScheduler-3] com.queue.SQSQueueWorker - x-company-requestid=sqs-worker service_name=orders_v3 Processing messages message_number=1
//--------------------------------------------
01 Aug 2016 04:48:15,600 INFO [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Received msg from SQS:QueueMessage [payload=428CB476547214700268914651663, delaySeconds=0, sqsId=7f4dcbbe-90c4-4e56-b4ab-50332597b5d8, stringAttributes={features=FIS-JEM, accept-language=EN_US, request-id=a2c31da4-517f-40ec-8587-624f97393659}]
I imagine some sort of regex would be needed, but I have no experience in using regex to do such a split through Java code.
I also found the following related question, but I didn't understand the solution suggested there:
java regex: capture multiline sequence between tokens
The following code uses the log you supplied and produces your desired output. I've included comments in the code that explain the program flow.
Code:
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.regex.Pattern;
public class Driver {
private static final String LINE_BREAK = System.lineSeparator();
// path to the logfile
private static final String LOG_FILE = "xin/xin.log";
// The regex below matches lines that begin with "00 Mth 0000 00:00:00,000".
private static Pattern pattern = Pattern.compile("^[0-9][0-9]\\s[A-Z][a-z][a-z]\\s[0-9][0-9][0-9][0-9]\\s[0-9][0-9]:[0-9][0-9]:[0-9][0-9],[0-9][0-9][0-9]");
public static void main(String[] args) {
BufferedReader reader = null;
StringBuilder sb = new StringBuilder();
String line = "";
boolean firstRun = true;
try {
// instantiated the buffered reader
reader = new BufferedReader(new FileReader(new File(LOG_FILE)));
} catch(FileNotFoundException e) {
e.printStackTrace();
}
// loop until all lines are read from log file
while(true) {
try {
// get next line from log file
line = reader.readLine();
// if no more content in log file then break out of loop
if(line == null) {
break;
}
} catch (IOException e) {
e.printStackTrace();
}
// we don't want a line-break before the first line
if(firstRun) {
firstRun = false;
// append the first line to the string builder
sb.append(line);
} else {
// we've handled the first line, so now we
// append a line-break to the string builder
// before appending the next line
sb.append(LINE_BREAK);
// if the line matches the timestamp pattern then
// append another line-break
if(pattern.matcher(line).find()) {
sb.append(LINE_BREAK);
}
// add the line to the string builder
sb.append(line);
}
}
try {
// close the buffered reader
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
// print string builder contents to standard out
System.out.println(sb.toString());
}
}
Output:
01 Aug 2016 04:48:13,311 ERROR [pool-2-thread-12436] com.orders.queue.OrdersQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Exception while calling /purchases: x-company-status : UNKNOWN_ERROR response status: Internal Server Error
01 Aug 2016 04:48:13,311 WARN [pool-2-thread-12436] com.orders.queue.OrdersQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Failed to process order, will be re-tried: ADD2500051FR
01 Aug 2016 04:48:13,332 INFO [pool-2-thread-12436] com.delegate - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Getting Email from Primary email
01 Aug 2016 04:48:13,363 WARN [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Message processing failed QueueMessage [payload=ADD2500051FR, delaySeconds=0, sqsId=51f70e3f-554a-463b-8384-0b2c25a90450, stringAttributes={features=adac2911-0578-4bcd-b8c3-783481a48e1d, accept-language=FR_FR, request-id=836ac8b6-515d-4414-b4c6-ddd8a52ef497}]
com.orders.exception.orderserviceException: Error in Calling PUT purchase from main service
at com.OrderServiceDelegate.handleInternalServerErrors(OrderServiceDelegate.java:352)
at com.OrderServiceDelegate.sendOrderForProcessing_aroundBody0(OrderServiceDelegate.java:113)
at com.OrderServiceDelegate.sendOrderForProcessing_aroundBody1$advice(OrderServiceDelegate.java:37)
at com.OrderServiceDelegate.sendOrderForProcessing(OrderServiceDelegate.java:1)
at com.orders.queue.OrdersQueueWorker.doWork(OrdersQueueWorker.java:168)
at com.queue.SQSQueueWorker.lambda$0(SQSQueueWorker.java:149)
at com.queue.SQSQueueWorker.dt_access$492(SQSQueueWorker.java)
at com.queue.SQSQueueWorker$$dtt$$Lambda$8/852112146.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
01 Aug 2016 04:48:13,365 INFO [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Order will be re-tried after 300 seconds: ADD2500051FR
01 Aug 2016 04:48:15,600 INFO [myScheduler-3] com.queue.SQSQueueWorker - x-company-requestid=sqs-worker service_name=orders_v3 Processing messages message_number=1
01 Aug 2016 04:48:15,600 INFO [pool-2-thread-12436] com.queue.SQSQueueWorker - x-company-requestid=836ac8b6-515d-4414-b4c6-ddd8a52ef497-sqs service_name=orders_v3 Received msg from SQS:QueueMessage [payload=428CB476547214700268914651663, delaySeconds=0, sqsId=7f4dcbbe-90c4-4e56-b4ab-50332597b5d8, stringAttributes={features=FIS-JEM, accept-language=EN_US, request-id=a2c31da4-517f-40ec-8587-624f97393659}]
If you need any additional help, then just leave a comment on this answer and I'll try my best to help.