Recently I've been investigation one bug and faced interesting case with Spring Boot Rest API.
I have a java object which I return to the user:
#Data
#JsonInclude(JsonInclude.Include.NON_NULL)
#AllArgsConstructor
#NoArgsConstructor
#Builder
#ToString
public class TestModel {
private List<String> valid = new ArrayList<>();
private Map<String, String> invalid = new HashMap<>();
}
It's a simple object which returns a list of valid ids and a map which contain invalid id as a key and message why it happened as a value. Usually this map is empty or have multiple elements. And it works fine. But when this map has only one element we receive an exception:
org.springframework.http.converter.HttpMessageNotWritableException: Could not write JSON: Invalid name start character '6' (code 54) (name "627228f554e7655db21f2859"); nested exception is com.fasterxml.jackson.databind.JsonMappingException: Invalid name start character '6' (code 54) (name "627228f554e7655db21f2859") (through reference chain: TestModel["invalid"]->java.util.Collections$UnmodifiableMap["627228f554e7655db21f2859"])
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:296)
at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:104)
Caused by: com.fasterxml.jackson.core.JsonGenerationException: Invalid name start character '6' (code 54) (name "627228f554e7655db21f2859")
at com.fasterxml.jackson.dataformat.xml.util.StaxUtil.throwAsGenerationException(StaxUtil.java:51)
at com.fasterxml.jackson.dataformat.xml.ser.ToXmlGenerator.writeString(ToXmlGenerator.java:676)
at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFieldsUsing(MapSerializer.java:905)
... 120 common frames omitted
Caused by: javax.xml.stream.XMLStreamException: Invalid name start character '6' (code 54) (name "627228f554e7655db21f2859")
at com.fasterxml.aalto.out.XmlWriter.throwOutputError(XmlWriter.java:475)
at com.fasterxml.aalto.out.XmlWriter.reportNwfName(XmlWriter.java:386)
at com.fasterxml.aalto.out.ByteXmlWriter.verifyNameComponent(ByteXmlWriter.java:224)
at com.fasterxml.aalto.out.ByteXmlWriter.constructName(ByteXmlWriter.java:170)
at com.fasterxml.aalto.out.WNameTable.findSymbol(WNameTable.java:324)
at com.fasterxml.aalto.out.RepairingStreamWriter._writeStartOrEmpty(RepairingStreamWriter.java:340)
at com.fasterxml.aalto.out.RepairingStreamWriter.writeStartElement(RepairingStreamWriter.java:231)
at com.fasterxml.jackson.dataformat.xml.ser.ToXmlGenerator.writeString(ToXmlGenerator.java:667)
Exception message is longer so I omitted some and put here just the main of it.
The interesting case it that I can't reproduce this issue locally. I'm running it on my mac machine and it works fine. I receive a response body in postman:
{
"invalid": {
"627228f554e7655db21f2859": "Not valid."
}
}
But it fails on my environment which is docker container ubuntu java17
I've tried to update a jackson lib to the latest. Now we're using 2.10.3, but I tried the latest I found in maven repo: 2.14.1. It didn't help. I also tried to reproduce this issue on my local docker. But I couldn't. It works fine.
Maybe someone faced with such issue already?
Thanks in advance!
I recently encountered a similar situation where some code worked locally but didn't work when deployed on kubernetes. The problem was with regex but it might be the same cause that caused your issue. I was using a regex [^\*\?\h]* to check validity of some input. At some point I saw some input failed by my regex that should not have been failed. And locally it worked. So I printed out the way regex looked on kubernetes and it turned out that it was interpreted as [^*?h]* instead of [^\*\?\h]* my guess is that kubernetes itself runs on java so it interpreted regex as java string and I needed to add extra slash for double encoding for java and regex. So I changed my regex from [^\*\?\h]* to [^\\*\\?\\h]* and it worked. To my surprise it also worked locally as well.So my guess for your problem that it somehow doesn't recognize symbol " within your JSON. SO you will need to replace " with \" if my guess is correct. You can do it with custom json serializer in your server side I guess
Related
It is as simple as it looks.
How to split List of integers in apache camel without making camel complains when he tries to split the list<Integer> into Bytes instead!
The code looks like:
....
.transform().message(this::doSomeProcessing) // doSomeProcessing returns List<Integer>
.filter(simple("${body.size} != 0"))
.split(body())
.to(someRabbitMQExchange())
....
And here's the exception:
Caused by: java.lang.IllegalArgumentException: Could not convert number [9901400] of type [java.lang.Integer] to target class [java.lang.Byte]: overflow
The issue was in rabbitmq endpoint, it needed the message to be converted to json first.
so, I just added .marshal().json() before .to(someRabbitMQExchange()) and it solved the issue.
Try comment o avoid the filter, run de app and validate again, tell me that happen later of this.
I implemented a kinesis stream consumer client using node wrapper and getting this MultiLangDaemon execution error as shown below.
Starting MultiLangDaemon ... java.lang.IllegalArgumentException: No enum constant software.amazon.kinesis.common.InitialPositionInStream.TRIM_HORIZON at java.lang.Enum.valueOf(Enum.java:238) at software.amazon.kinesis.common.InitialPositionInStream.valueOf(InitialPositionInStream.java:21) at software.amazon.kinesis.multilang.config.MultiLangDaemonConfiguration$2.convert(MultiLangDaemonConfiguration.java:208) at org.apache.commons.beanutils.ConvertUtilsBean.convert(ConvertUtilsBean.java:491) at org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:1007) at software.amazon.kinesis.multilang.config.KinesisClientLibConfigurator.lambda$getConfiguration$0(KinesisClientLibConfigurator.java:65) at java.lang.Iterable.forEach(Iterable.java:75) at java.util.Collections$SynchronizedCollection.forEach(Collections.java:2064) at software.amazon.kinesis.multilang.config.KinesisClientLibConfigurator.getConfiguration(KinesisClientLibConfigurator.java:63) at software.amazon.kinesis.multilang.MultiLangDaemonConfig.<init>(MultiLangDaemonConfig.java:101) at software.amazon.kinesis.multilang.MultiLangDaemonConfig.<init>(MultiLangDaemonConfig.java:74) at software.amazon.kinesis.multilang.MultiLangDaemonConfig.<init>(MultiLangDaemonConfig.java:58) at software.amazon.kinesis.multilang.MultiLangDaemon.buildMultiLangDaemonConfig(MultiLangDaemon.java:171) at software.amazon.kinesis.multilang.MultiLangDaemon.main(MultiLangDaemon.java:220) No enum constant software.amazon.kinesis.common.InitialPositionInStream.TRIM_HORIZON
I already cross checked properties file with details shown below listed there
initialPositionInStream, processingLanguage,streamName,executableName etc.
TRIM_HORIZON is set as value for initialPositionInStream Not sure why software.amazon.kinesis.common.InitialPositionInStream this object missing this value?
I am using node consumer client as mentioned here
https://docs.aws.amazon.com/streams/latest/dev/kinesis-record-processor-implementation-app-nodejs.html
Just want to help community by answering this specific error fix. It was .properties file update for that respective parameter key name. Although I am getting couple of other java.lang.RuntimeException that will try to resolve and ask for resolution in separate threads.
I'm getting a "Conditionally Required Field Missing" error message, even though I'm sure that field is there.
58=Conditionally Required Field Missing, field=55
Versions:
QuickFixJ 2.1.0
FIX 4.4
Here is the FIX message that I'm sending (with mocked values and a few fields removed for clarity)
8=FIX.4.4
9=709
35=R
34=4
49=TARGET
56=ME
11=myClOrdID
131=myQuoteReqID
146=myNoRelatedSym
55=mySymbol // field missing
167=mySecurityType // field missing
Here is the calling code:
String symbol = quoteRequest.getField(new StringField(55)).getValue();
I also tried:
String symbol = quoteRequest.getString(55);
Here is my Data Dictionary:
<field number="55" name="Symbol" type="STRING"/>
I realize that the symbol field is no longer a part of the QuoteRequest FIX specification for 4.4 (though it was in earlier versions, such as 4.0), however surely there are ways to retrieve custom fields? I have no control over the QuoteRequest message that I receive.
I can always parse the message myself using toString() but that kinda defeats the purpose of using quickfixj in the first place.
Any ideas?
Tag 55 is inside the 146 repeating group. See the docs for reading repeating groups.
The symbol field is still in FIX44. You should spend some time familiarizing yourself with the FIX44.xml data dictionary file that you're using.
(You may find that you need to customize that file based on your counterparty's messaging; in practice, nobody uses the basic FIX44 message definitions without changing them at least a little.)
// create group
QuoteRequest.NoRelatedSym group = new QuoteRequest.NoRelatedSym();
// set group, confusing method name I find
message.getGroup(1, group);
// you now have all the getters of fields in that group
Symbol symbol = group.getSymbol();
I'm trying to check whether a specific AMI exists. So, I'm doing:
val filter = new Filter().withName("Name").withValues(amiName)
val result = ec2.describeImages(new DescribeImagesRequest().withFilters(filter))
result.getImages.size() > 0
(code is Scala and not Java, but that's not really relevant). I'm getting the following exception:
com.amazonaws.AmazonServiceException: The filter 'Name' is invalid
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:644) ~[aws-java-sdk-1.4.2.1.jar:na]
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:338) ~[aws-java-sdk-1.4.2.1.jar:na]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:190) ~[aws-java-sdk-1.4.2.1.jar:na]
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:6199) ~[aws-java-sdk-1.4.2.1.jar:na]
at com.amazonaws.services.ec2.AmazonEC2Client.describeImages(AmazonEC2Client.java:2905) ~[aws-java-sdk-1.4.2.1.jar:na]
How do I correctly define a filter to the DescribeImagesRequest?
TL;DR - use name instead of Name as the key.
To investigate, I turned to my local installation of ec2 tools, and ran
ec2-describe-images -o self -F name=myaminame
Got a similar error that was more Google friendly:
Filter definitions must have format 'name=value', but found 'name'
Googlging got me to this blog post, and afterwards this worked from the command line:
ec2-describe-images -o self -F "name=myaminame"
Now, after this unrelated excursion, I just found the simple problem: I tried Name as key, while in fact the key should be lowercase name.
We are getting a mustache play error in production (amazon linux EC2 AMI) but not in development (MACs) and we have tried upgrading the jvm, using the jdk instead, and changing from a tomcat deploy model to match our development environments as much as possible but nothing is working. Please any help would be greatly appreciated. We have lots of shared code in java and javascript using mustache and it would be a big deal to rewrite everything if we had to ditch mustache on the java side.
20:48:52,403 ERROR ~
#6al2dd0po
Internal Server Error (500) for request GET /mystuff/people
Execution exception (In {module:mustache-0.2}/app/play/modules/mustache/MustacheTags.java around line 32)
NullPointerException occured : null
play.exceptions.JavaExecutionException
at play.templates.BaseTemplate.throwException(BaseTemplate.java:90)
at play.templates.GroovyTemplate.internalRender(GroovyTemplate.java:257)
at play.templates.Template.render(Template.java:26)
at play.templates.GroovyTemplate.render(GroovyTemplate.java:187)
at play.mvc.results.RenderTemplate.<init>(RenderTemplate.java:24)
at play.mvc.Controller.renderTemplate(Controller.java:660)
at play.mvc.Controller.renderTemplate(Controller.java:640)
at play.mvc.Controller.render(Controller.java:695)
at controllers.MyStuff.people(MyStuff.java:183)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:548)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:502)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:478)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:473)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:161)
at Invocation.HTTP Request(Play!)
Caused by: java.lang.NullPointerException
at play.modules.mustache.MustacheTags._template(MustacheTags.java:32)
at play.modules.mustache.MustacheTags$_template.call(Unknown Source)
at /app/views/User/people.html.(line:22)
at play.templates.GroovyTemplate.internalRender(GroovyTemplate.java:232)
... 13 more
Seems the issue is with the threadlocal. In Prod as per my logs, the session gets initialized with the main thread.
[2012-06-30 18:35:38,102] INFO 10097[**main**] - Mustache module initialized
However, MustacheTag tries to access with various thread like this during request.
[2012-06-30 17:48:44,669] INFO 66048[**play-thread-1**] - [{module:mustache-0.2}/app/play/modules/mustache/MustacheTags.java:46] _meta() :: MustachePlugin.session():null
So I changed the implementation of MustachePlugin like this.Changed line commented out:
//private static ThreadLocal<MustacheSession> session_ = new ThreadLocal<MustacheSession>();
private static MustacheSession _session = null;
public static MustacheSession session(){
//return session_.get();
return _session;
}
public void onConfigurationRead(){
// some code
_session = new MustacheSession(compiler, root);
// some code
}
And it is working fine now in prod mode! I see no reason why it should have been in a ThreadLocal in the first place as the session gets initialized at startup!
your issue is difficult to reproduce so i'll give few pointers here. you have tried to eliminate of issue being env issue. so other possible issues could be
data issue: many times reason for production issue is usually difference in actual data and test data. check if its data issue which is casuing NPE.
code issue: Is there something at people.html.(line:22) causing issue. first try removing / altering that to check if thats causing issue. Or can you get source code of mustache (exact version which you are using) and see what object its trying to create and where its failing.
Properties file for different environments: do you have different proprties file for each env? If yes, have u missed on any property for prod env?
You have a NullPointerException on MustacheTags.java at line 32.
This means that you are probably calling a method of the Mustache library and passing a null value.
Try logging all the parameters you transfer to this method (MyStuff.java line 183?).
You can look at the source code of MustacheTags here, it might help you understand what values you are passing and what should be passed.