I wrote JUnit test for the web service client, which submits JSON document to the service.
I saved "correct" JSON document to the file, then after the test execution I compare it with actual result.
They are not matched, although lines are identical:
org.junit.ComparisonFailure:
Expected :{"Callback":null,"Data":
{"MarketCode":"ISEM",,............"Price":2.99}]}]}]}]}}
Actual :{"Callback":null,"Data":
{"MarketCode":"ISEM",,............"Price":2.99}]}]}]}]}}
Lines are very long , about 4K characters, so I cut much of it here, but their length is identical. I compared string.size() in the debugger , and also I trim it before the compare, to remove some invisible or whitespace symbols in the end, which text editor can implicitly insert.
Also, test is OK when executed isolately. But it fails , when I run it as part of bigger suite.
There is no global/static variables, so memory overriding should be not an issue.
I'm mocking web service client to extract the request string , like this:
StringBuilder pd = new StringBuilder();
doAnswer((invocation) -> {
String postDocument = ((String)invocation.getArguments()[0]).trim();
pd.append(postDocument);
return null;
}).when(client).doPost(anyString(), anyObject());
client is mocked class.
Then I compare trimmed versions of strings, but it doesnt help
String expectedSubmit = TestUtils.readXmlFromFile("strategyexecution\\ireland_bm_strategy_override_expected.json").trim();
assertEquals(expectedSubmit, pd.toString().trim());
I found answer myself :-)
The problem is with JSON specification itself.
JSON cannot guarantee the same order of elements inside the array, it's basically unordered set.
So, the content can be randomly reordered. Two produced JSON files should not be compared as two strings.
I deserialized it to Java object and object comparision works!
Same old issue as we had with XML. For XML there is XMLUnit which semantically compares xml-s. For JSON I'd try to use a similar tool, like JsoNunit. JSONAssert too looks promising.
Related
I am receiving a JSON in the form of a string, need to mask a piece of information, however the JSON strucutre and key-names are always different but value's pattern is recognizable. Question being, what is an efficient way to traverse through String/JSONObject to mask that piece of data.
I've tried turning the String into a JSONObject and traverse through every embedded JSONObject/Array, detect the pattern, and replace that original value with its masked version. But this seems very time consuming when Logging this information out to console.
Value's pattern for reference is a 9-digit (Long) number.
Structure always varies from "{"key1":[{"innerKey1":123456789}]}" to "{"key1":"value1", "key2":{"innerKey1":123456789}"
Sample result : "{"key1":[{"innerKey1":"XXXXXX789"}]}"
If the JSON structure is always provided as optimized single line string you could just find the value in the string and replace it or get even more elaborate and use a regular expression to find the innerKey1:12345 match and replace that.
If this is just for logging purposes you can even implement this as a filter, depending on your logging framework it might be even configurable instead of having to code it.
i am looking to find an example of Java-diff-utils finding the difference between two strings e.g.
String s1 = "{\"type\": \"message_typing\",\"requestId\": \"requestid\",\"clientMutationId\": \"mutationid\",\"chatRoomId\": \"roomid\",\"conversationId\": \"conversationid\"}";
String s2 = "{\"type\": \"type_2\",\"requestId\": \"1\",\"clientMutationId\": \"2\",\"chatRoomId\": \"dev/test\",\"conversationId\": \"aa2344ceDsea1\"}";
so the first is the base message the second is the one i would like to compare against the base message and get the different values (e.g. type_2,1,2,dev/test,aa2344ceDsea1) however i would like to be able to reassemble the string correctly if given only the base message and the values of the diffs.
I can only find examples using two files online and no examples using two strings if anyone could give me a code example that would be very helpful. I have already did it using google-diff-patch-match however the returned diffs are too large for what i need. as i will be sending the diffs over MQTT in order to keep payload size down, so i need something that can extract the values but still be able to reassemble on the other side when given the values and base message.
I have a class called GamesData that has strings and getters and setters for this strings.
I download strings from a json. One of these strings is an URL with an image. I download the images but on a small size, because of their URL. I need to download it on a bigger size. For this, I need to remove this "small" string from the URL:
"home_team_logo": "https:\/\/URL\/images\/teams\/small\/olympique-marseille-890.png"
I have more than one URL coming from a big json object, all inside a json array, formated like the one above.
This is what I do to get the json.
arrayList.add(new GamesData(
gamesDataObject.getString(TAG_DATE),
gamesDataObject.getString(TAG_COMPETITION),
gamesDataObject.getString(TAG_HOME_TEAM),
gamesDataObject.getString(TAG_AWAY_TEAM),
gamesDataObject.getString(TAG_ID)
));
I need to remove from TAG_HOME_TEAM
public static final String TAG_HOME_TEAM= "home_team_logo";
, which is that URL above, only the "small" part, so the image downloaded will be the full size one.
In fact I will need this for all 3 tags: TAG_COMPETITION, TAG_HOME_TEAM, TAG_AWAY_TEAM.
How on earth do I do this? xD
You can use replace , use /small/ with / as replacement to avoid matching something like /othersmallteamname/
String s = "https:/URL/images/teams/small/olympique-marseille-890.png";
System.out.println(s.replace("/small/", "/"));
Output :
https:/URL/images/teams/olympique-marseille-890.png
I'm thinking something like this might work:
gamesDataObject.getString(TAG_HOME_TEAM).replaceAll("\/small\/","/");
This uses a regular expression to match all occurrences of the patern. A simplepr less error prone approach may be to use
gamesDataObject.getString(TAG_HOME_TEAM).replace("\/small\/","/");
Which should only replace the first occurrence.
I have a file that contains 10 lines - I want to retrieve it, and then split them with a newline("\n") delimiter.
here's what I did
val data = io.Source.fromFile("file.txt").toString;
But this causes an error when I try to split the file on newlines.
I then tried
val data = io.Source.fromFile("file.txt").mkString;
And it worked.
What the heck? Can someone tell me what the difference between the two methods are?
Let's look at the types, shall we?
scala> import scala.io._
import scala.io._
scala> val foo = Source.fromFile("foo.txt")
foo: scala.io.BufferedSource = non-empty iterator
scala>
Now, the variable that you have read the file foo.txt into is an iterator. If you perform toString() invocation on it, it doesn't return the contents of the file, rather the String representation of the iterator you've created. OTOH, mkString() reads the iterator(that is, iterates over it) and constructs a long String based on the values read from it.
For more info, look at this console session:
scala> foo.toString
res4: java.lang.String = non-empty iterator
scala> res4.foreach(print)
non-empty iterator
scala> foo.mkString
res6: String =
"foo
bar
baz
quux
dooo
"
scala>
The toString method is supposed to return the string representation of an object. It is often overridden to provide a meaningful representation. The mkString method is defined on collections and is a method which joins the elements of the collection with the provided string. For instance, try something like:
val a = List("a", "b", "c")
println(a.mkString(" : "))
and you will get "a : b : c" as the output. The mkString method has created a string from your collection by joining the elements of the collection with the string you provided. In the particular case you posted, the mkString call joined the elements returned by the BufferedSource iterator with the empty string (this is because you called mkString with no arguments). This results in simply concatenating all of the strings (yielded by the BufferedSource iterator) in the collection together.
On the other hand, calling toString here doesn't really make sense, as what you are getting (when you don't get an error) is the string representation of the BufferedSource iterator; which just tells you that the iterator is non-empty.
They're different methods in different classes. In this case, mkString is a method in the trait GenTraversableOnce. toString is defined on Any (and is very often overridden).
The easiest way (or at least the way I usually use) to find this out is to use the documentation at http://www.scala-lang.org/api/current/index.html. Start with the type of your variable:
val data = io.Source.fromFile("file.txt")
is of type
scala.io.BufferedSource
Go to the doc for BufferedSource, and look for mkString. In the doc for mkString (hit the down arrow over to the left) you'll see that it comes from
Definition Classes TraversableOnce → GenTraversableOnce
And do the same thing with toString.
I think the problem is to understand what Source class is doing. It seems from your code that you expect that Source.fromFile retrieves the content of a file when really what it does is to point to the start of a file.
This is typical when working with I/O operations where you have to open a "connection" with a resource (on this case a connection with your filesystem), read/write several times and then close that "connection". In your example you open a connection to a file and you have to read line per line the contents of the file until you reach the end. Think that when you read you are loading information in memory so it's not a good idea to load the whole file in memory in most of the scenarios (which mkString is going to do).
In the other hand mkString is made to iterate over all the elements of a collection, so in this case what is does is to read the file and load an Array[String] in memory. Be careful because if the file is big your code will fail, normally when working with I/O you should use a buffer to read some content, then process/save that content and then load more content (in the same buffer), avoiding problems with memory. For example reading 5 lines --> parse --> save parsed lines --> read next 5 lines --> etc.
You can also understand that "toString" retrieves you nothing... just tells you "you can read lines, the file is not empty".
So,
I'm trying to convert a Representation to a String or a StringWriter either using the getText() or write() method. It seems I can only call this method once successfully on a Representation... If I call the method again, it returns null or empty string on the second call. Why is this? I'd expect it to return the same thing every time:
public void SomeMethod(Representation rep)
{
String repAsString = rep.getText(); // returns valid text for example: <someXml>Hello WOrld</someXml>
String repAsString2 = rep.getText(); // returns null... wtf?
}
If I'm "doing it wrong" then I'd be open to any suggestions as to how I can get to that data.
The javadocs explain this:
The content of a representation can be
retrieved several times if there is a
stable and accessible source, like a
local file or a string. When the
representation is obtained via a
temporary source like a network
socket, its content can only be
retrieved once.
So presumably it's being read directly from the network or something similar.
You can check this by calling isTransient(). If you need to be able to read it multiple times, presumably you should convert it to a string and then create a new Representation from that string.
It's because in general the Representation doesn't actually get read in from the InputStream until you ask for it with getText(), and once you've asked for it, all the bytes have been read and converted into the String.
This is the natural implementation for efficiency: rather than creating a potentially very large String and then converting that String into something useful (a JSON object, a DOM tree, or whatever), you write your converter to operate on the InputStream instead, avoiding the costs of making and reading that huge String.
So for example if you have a large XML file being PUT into a web service, you can feed the InputStream right into a SAX parser.
(As #John notes, a StringRepresentation wraps a String, and so can be read multiple times. But you must be reading a Request's representation, which is most likely an InputRepresentation.)