I am trying to read a csv file into JavaRDD. In order to do that, I wrote the code below:
SparkConf conf = new SparkConf().setAppName("NameOfApp").setMaster("spark://Ip here:7077");
JavaSparkContext sc = new JavaSparkContext(conf);
JavaRDD<CurrencyPair> rdd_records = sc.textFile(System.getProperty("user.dir") + "/data/data.csv", 2).map(
new Function<String, CurrencyPair>() {
public CurrencyPair call(String line) throws Exception {
String[] fields = line.split(",");
CurrencyPair sd = new CurrencyPair(Integer.parseInt(fields[0].trim()), Double.parseDouble(fields[1].trim()),
Double.parseDouble(fields[2].trim()), Double.parseDouble(fields[3]), new Date(fields[4]));
return sd;
}
}
);
My data file looks like this:
1,0.034968,212285,7457.23,"2019-03-08 18:36:18"
Here, in order to check that if my data loaded correctly or not, I tried to print some of them:
System.out.println("Count: " + rdd_records.count());
List<CurrencyPair> list = rdd_records.top(5);
System.out.println(list.toString());
But I had following error at both system out lines. I tried each of them alone as well rather than printing count and list at the same time.
Caused by: java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f of type scala.Function3 in instance of org.apache.spark.rdd.MapPartitionsRDD
My custom object looks like this:
public class CurrencyPair implements Serializable {
private int id;
private double value;
private double baseVolume;
private double quoteVolume;
private Date timeStamp;
public CurrencyPair(int id, double value, double baseVolume, double quoteVolume, Date timeStamp) {
this.id = id;
this.value = value;
this.baseVolume = baseVolume;
this.quoteVolume = quoteVolume;
this.timeStamp = timeStamp;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public double getValue() {
return value;
}
public void setValue(double value) {
this.value = value;
}
public double getBaseVolume() {
return baseVolume;
}
public void setBaseVolume(double baseVolume) {
this.baseVolume = baseVolume;
}
public double getQuoteVolume() {
return quoteVolume;
}
public void setQuoteVolume(double quoteVolume) {
this.quoteVolume = quoteVolume;
}
public Date getTimeStamp() {
return timeStamp;
}
public void setTimeStamp(Date timeStamp) {
this.timeStamp = timeStamp;
}
}
So I could not figured out what is wrong here. What am I doing wrong?
Edit: It works well when I write local instead of my own spark master IP. But I need to run this on my own IP. So what can be wrong with my master node?
The issue is probably the anonymous class definition new Function<String, CurrencyPair>() { which forces Spark to try to serialize the parent class as well. Try a lambda instead:
rdd_records.map(
(Function<String, CurrencyPair>) line -> {
...
Note: You could read the file as a CSV instead and use the dataset API with a bean encoder to skip the manual parsing completely.
Related
I have a custom object :
public class Response
{
#JsonProperty("values")
private List<Value> values;
#JsonFormat(shape=JsonFormat.Shape.ARRAY)
public static class Value {
public Value(long timestamp, float val)
{
this.timestamp = timestamp;
this.val = val;
}
}
}
But when this is being parsed, I get "Can not deserialize instance of Response$Value out of START_ARRAY token".
The json is :
{
"values":[[1552215648,18]]
}
Any idea if I'm missing something here? Or should I have a custom deserializer to perform this?
JsonFormat does the trick but you also need to declare constructor with JsonCreator annotation. Take a look on below example:
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.File;
import java.util.List;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = new ObjectMapper();
Response myPojo = mapper.readValue(jsonFile, Response.class);
System.out.println(myPojo);
}
}
class Response {
private List<Value> values;
public List<Value> getValues() {
return values;
}
public void setValues(List<Value> values) {
this.values = values;
}
#Override
public String toString() {
return "Response{" +
"values=" + values +
'}';
}
}
#JsonFormat(shape = JsonFormat.Shape.ARRAY)
class Value {
private long timestamp;
private float val;
#JsonCreator
public Value(#JsonProperty("timestamp") long timestamp, #JsonProperty("val") float val) {
this.timestamp = timestamp;
this.val = val;
}
public long getTimestamp() {
return timestamp;
}
public void setTimestamp(long timestamp) {
this.timestamp = timestamp;
}
public float getVal() {
return val;
}
public void setVal(float val) {
this.val = val;
}
#Override
public String toString() {
return "Value{" +
"timestamp=" + timestamp +
", val=" + val +
'}';
}
}
Above code for below JSON payload:
{
"values": [
[
1552215648,
18
],
[
123,
12.24
]
]
}
Prints:
Response{values=[Value{timestamp=1552215648, val=18.0}, Value{timestamp=123, val=12.24}]}
You should JsonFormat over the variable values. Also since you are having the variable as of type List, you need not add JsonFormat.
public class Response
{
#JsonProperty("values")
#JsonFormat(shape=JsonFormat.Shape.ARRAY)
private List<Value> values;
public static class Value {
private long timestamp;
private float val;
// Getters and Setters
public Value(long timestamp, float val)
{
this.timestamp = timestamp;
this.val = val;
}
}
}
Your JSON input format would be:
{
values: [
{
"timestamp": 589988,
"val": 56.0,
}
]
}
Hope this helps !! I haven't tested the code, so please ignore syntax issues if any.
I know it's been a while, but since I dealt with the same problem (deserializing part of Prometheus REST API response?), and provided answers weren't helping, I would like to share my solution which should work for every similar case.
Apart from #JsonFormat(shape=JsonFormat.Shape.ARRAY) it was necessary for me to add also #JsonPropertyOrder annotation to reflect the order in which values appear in the array. I also needed #JsonCreator annotation on constructor and #JsonProperty before every constructor's argument. Using your class as an example it should look like this:
public class Response
{
private List<Value> values;
#JsonFormat(shape=JsonFormat.Shape.ARRAY)
#JsonPropertyOrder({"timestamp", "val"})
public static class Value {
private long timestamp;
private float val;
#JsonCreator
public Value(#JsonProperty("timestamp") long timestamp,
#JsonProperty("val") float val)
{
this.timestamp = timestamp;
this.val = val;
}
// Getters and Setters
}
}
Only after I used this particular set of annotations my response was deserialised without any errors.
Hope this is gonna help someone! ;)
EDIT: One note: in Prometheus response I got, an array of values of different types was called value and not values, hence it might be necessary to add a #JsonProperty("value") on private List<Value> values; field or change it's name to value if that is the case for anyone ;)
#JsonProperty("value")
private List<Value> values;
I have a custom object :
public class Response
{
#JsonProperty("values")
private List<Value> values;
#JsonFormat(shape=JsonFormat.Shape.ARRAY)
public static class Value {
public Value(long timestamp, float val)
{
this.timestamp = timestamp;
this.val = val;
}
}
}
But when this is being parsed, I get "Can not deserialize instance of Response$Value out of START_ARRAY token".
The json is :
{
"values":[[1552215648,18]]
}
Any idea if I'm missing something here? Or should I have a custom deserializer to perform this?
JsonFormat does the trick but you also need to declare constructor with JsonCreator annotation. Take a look on below example:
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.File;
import java.util.List;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = new ObjectMapper();
Response myPojo = mapper.readValue(jsonFile, Response.class);
System.out.println(myPojo);
}
}
class Response {
private List<Value> values;
public List<Value> getValues() {
return values;
}
public void setValues(List<Value> values) {
this.values = values;
}
#Override
public String toString() {
return "Response{" +
"values=" + values +
'}';
}
}
#JsonFormat(shape = JsonFormat.Shape.ARRAY)
class Value {
private long timestamp;
private float val;
#JsonCreator
public Value(#JsonProperty("timestamp") long timestamp, #JsonProperty("val") float val) {
this.timestamp = timestamp;
this.val = val;
}
public long getTimestamp() {
return timestamp;
}
public void setTimestamp(long timestamp) {
this.timestamp = timestamp;
}
public float getVal() {
return val;
}
public void setVal(float val) {
this.val = val;
}
#Override
public String toString() {
return "Value{" +
"timestamp=" + timestamp +
", val=" + val +
'}';
}
}
Above code for below JSON payload:
{
"values": [
[
1552215648,
18
],
[
123,
12.24
]
]
}
Prints:
Response{values=[Value{timestamp=1552215648, val=18.0}, Value{timestamp=123, val=12.24}]}
You should JsonFormat over the variable values. Also since you are having the variable as of type List, you need not add JsonFormat.
public class Response
{
#JsonProperty("values")
#JsonFormat(shape=JsonFormat.Shape.ARRAY)
private List<Value> values;
public static class Value {
private long timestamp;
private float val;
// Getters and Setters
public Value(long timestamp, float val)
{
this.timestamp = timestamp;
this.val = val;
}
}
}
Your JSON input format would be:
{
values: [
{
"timestamp": 589988,
"val": 56.0,
}
]
}
Hope this helps !! I haven't tested the code, so please ignore syntax issues if any.
I know it's been a while, but since I dealt with the same problem (deserializing part of Prometheus REST API response?), and provided answers weren't helping, I would like to share my solution which should work for every similar case.
Apart from #JsonFormat(shape=JsonFormat.Shape.ARRAY) it was necessary for me to add also #JsonPropertyOrder annotation to reflect the order in which values appear in the array. I also needed #JsonCreator annotation on constructor and #JsonProperty before every constructor's argument. Using your class as an example it should look like this:
public class Response
{
private List<Value> values;
#JsonFormat(shape=JsonFormat.Shape.ARRAY)
#JsonPropertyOrder({"timestamp", "val"})
public static class Value {
private long timestamp;
private float val;
#JsonCreator
public Value(#JsonProperty("timestamp") long timestamp,
#JsonProperty("val") float val)
{
this.timestamp = timestamp;
this.val = val;
}
// Getters and Setters
}
}
Only after I used this particular set of annotations my response was deserialised without any errors.
Hope this is gonna help someone! ;)
EDIT: One note: in Prometheus response I got, an array of values of different types was called value and not values, hence it might be necessary to add a #JsonProperty("value") on private List<Value> values; field or change it's name to value if that is the case for anyone ;)
#JsonProperty("value")
private List<Value> values;
I'm running into a few issues similar to what others have had in the past with Json parsing in Java. This is the first time I try something like this so any help/tips is extremely useful.
I'm trying to parse in data from this site: https://api.bitcoinaverage.com/exchanges/USD
I have tried numerous ways with both Json and Gson. And have tried looking for help here but to no avail.
Here are the classes that are set up (these were auto generated):
Info.java:
public class Info{
private String display_URL;
private String display_name;
private Rates[] rates;
private String source;
private Number volume_btc;
private Number volume_percent;
public String getDisplay_URL(){
return this.display_URL;
}
public void setDisplay_URL(String display_URL){
this.display_URL = display_URL;
}
public String getDisplay_name(){
return this.display_name;
}
public void setDisplay_name(String display_name){
this.display_name = display_name;
}
public Rates[] getRates(){
return this.rates;
}
public void setRates(Rates[] rates){
this.rates = rates;
}
public String getSource(){
return this.source;
}
public void setSource(String source){
this.source = source;
}
public Number getVolume_btc(){
return this.volume_btc;
}
public void setVolume_btc(Number volume_btc){
this.volume_btc = volume_btc;
}
public Number getVolume_percent(){
return this.volume_percent;
}
public void setVolume_percent(Number volume_percent){
this.volume_percent = volume_percent;
}
}
Rates.java:
public class Rates {
private Number ask;
private Number bid;
private Number last;
public Number getAsk(){
return this.ask;
}
public void setAsk(Number ask){
this.ask = ask;
}
public Number getBid(){
return this.bid;
}
public void setBid(Number bid){
this.bid = bid;
}
public Number getLast(){
return this.last;
}
public void setLast(Number last){
this.last = last;
}
}
MainClass.java:
public class MainClass {
public static void main(String[] args) throws Exception {
Gson gson = new Gson();
String json = readUrl("https://api.bitcoinaverage.com/exchanges/USD");
Info page = gson.fromJson(json, Info.class);
System.out.println(page.getDisplay_name());
}
private static String readUrl(String urlString) throws Exception {
BufferedReader reader = null;
try {
URL url = new URL(urlString);
reader = new BufferedReader(new InputStreamReader(url.openStream()));
StringBuffer buffer = new StringBuffer();
int read;
char[] chars = new char[1024];
while ((read = reader.read(chars)) != -1)
buffer.append(chars, 0, read);
return buffer.toString();
} finally {
if (reader != null)
reader.close();
}
}
}
When I try to call a getter, a null is returned.
How do I go about parsing the data properly, and then being able to call an attribute from which ever object I want? For example, if I want an attribute from "anx_hk" or "bitfinex".
This is the first time me posting something here so I hope I'm following the proper guidelines.
I also plan on passing this over to Android once I get the fell for parsing Json better. Thanks for the help! It'll greatly be appreciated.
I'll be honest with you, that's a pretty lame API response. Here it is
{
"anx_hk": {
"display_URL": "https://anxbtc.com/",
"display_name": "ANXBTC",
"rates": {
"ask": 454.26,
"bid": 444.46,
"last": 443.78
},
"source": "bitcoincharts",
"volume_btc": 11.73,
"volume_percent": 0.02
},
...,
"timestamp": "Fri, 04 Apr 2014 04:30:26 -0000",
...
}
There's no JSON array here, so you can get rid of all your array types. This response is a JSON object, which contains a bunch of JSON objects (which share a format) and a JSON name value pair where the name is timestamp.
The common JSON objects have two fields of type double (that's what type your field should be, not Number)
"volume_btc": 11.73,
"volume_percent": 0.02
, three fields of type String
"display_URL": "https://anxbtc.com/",
"display_name": "ANXBTC",
"source": "bitcoincharts",
and one that is a JSON object that contains three more doubles
"rates": {
"ask": 454.26,
"bid": 444.46,
"last": 443.78
}
The actual issue here is that, I'm assuming, the JSON objects in the root JSON object have names that may change or new ones may be added. This is not a good fit for a POJO. Instead you'd want to use a Map<String, Info>, but Gson can't map to that by default. It is not well suited for such deserialization. You'd have to provide your own TypeAdapter.
Instead, I'm going to suggest you use Jackson.
If we put that all together, we get something like
class ApiResponse {
private Map<String, Info> page = new HashMap<>();
private Date timestamp;
public Map<String, Info> getPage() {
return page;
}
#JsonAnySetter
public void setPage(String name, Info value) {
page.put(name, value);
}
public Date getTimestamp() {
return timestamp;
}
public void setTimestamp(Date timestamp) {
this.timestamp = timestamp;
}
}
class Info {
private String display_URL;
private String display_name;
private Rates rates;
private String source;
private Double volume_btc;
private Double volume_percent;
public String getDisplay_URL() {
return this.display_URL;
}
public void setDisplay_URL(String display_URL) {
this.display_URL = display_URL;
}
public String getDisplay_name() {
return this.display_name;
}
public void setDisplay_name(String display_name) {
this.display_name = display_name;
}
public Rates getRates() {
return this.rates;
}
public void setRates(Rates rates) {
this.rates = rates;
}
public String getSource() {
return this.source;
}
public void setSource(String source) {
this.source = source;
}
public Double getVolume_btc() {
return this.volume_btc;
}
public void setVolume_btc(Double volume_btc) {
this.volume_btc = volume_btc;
}
public Double getVolume_percent() {
return this.volume_percent;
}
public void setVolume_percent(Double volume_percent) {
this.volume_percent = volume_percent;
}
}
class Rates {
private Double ask;
private Double bid;
private Double last;
public Number getAsk() {
return this.ask;
}
public void setAsk(Double ask) {
this.ask = ask;
}
public Double getBid() {
return this.bid;
}
public void setBid(Double bid) {
this.bid = bid;
}
public Double getLast() {
return this.last;
}
public void setLast(Double last) {
this.last = last;
}
}
With deserialization code such as
String json = readUrl("https://api.bitcoinaverage.com/exchanges/USD");
ObjectMapper mapper = new ObjectMapper();
ApiResponse response = mapper.readValue(json, ApiResponse.class);
System.out.println(response);
With appropriate toString() methods (mine were auto-generated with Eclipse), you would get something like
ApiResponse [pages={bitkonan=Info [display_URL=https://bitkonan.com/, display_name=BitKonan, rates=Rates [ask=475.0, bid=438.01, last=437.0], source=api, volume_btc=7.24, volume_percent=0.01], vaultofsatoshi=Info [display_URL=https://vaultofsatoshi.com, display_name=Vault of Satoshi, rates=Rates [ask=460.0, bid=460.0, last=460.0], source=api, volume_btc=11.46, volume_percent=0.02], bitstamp=Info [display_URL=https://bitstamp.net/, display_name=Bitstamp, rates=Rates [ask=439.16, bid=436.34, last=436.34], source=api, volume_btc=22186.29, volume_percent=35.19], ...}, timestamp=Fri Apr 04 01:02:43 EDT 2014]
as output.
The api response contains many objects, but seems that you are trying to read them as a single Info object.
You may try to read the response as a Map<String, Info>, and iterate the entries.
Map<String, Info> hashMap = gson.fromJson(body, HashMap.class);
for (Map.Entry entry : hashMap.entrySet()) {
// your code
}
I am trying to show DB data to my webpage.
I have made following code when GET request to the #RequestMapping(value = "/api/binder").
but when get request came to this method it will fetch data (I have print on console and display well) but it doesn't map to my Java Script Ajax call, it's showing me an error.
Following is my code for to fetch data :
#Autowired
IBinderViewRepository repository;
#RequestMapping(method= RequestMethod.GET)
public #ResponseBody
List<BinderResponse> getBinders(){
List<BinderView> binders = repository.getBinders();
List<BinderResponse> responses = new ArrayList<>();
ModelMapper mapper = Mapper.getInstance();
for(int i = 0; i < binders.size(); i++){
System.out.println("In Loop");
BinderResponse response = mapper.map(binders.get(i),BinderResponse.class);
System.out.println("Data :: " + response.getBinderName());
responses.add(response);
}
return responses;
}
but it shows me following error :
HTTP Status 500 - Could not write JSON: (was java.lang.NullPointerException) (through reference chain: java.util.ArrayList[0]->com.ngl.dto.outgoing.BinderResponse["valid"]); nested exception is com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: java.util.ArrayList[0]->com.ngl.dto.outgoing.BinderResponse["valid"])
Here is ajax call from knockout js :
ajax.get('api/binder').done(function(response){ ... }
Here BinderView and BinderResponse have same fields :
private String binderName;
private String binderAddress1;
and getter setter as well in both.
and repository.genBinders() method bring data from DB.
Here is insert method and works fine for me :
#RequestMapping(method= RequestMethod.POST,consumes = "application/json")
public #ResponseBody
IWebApiResponse addBinder(#RequestBody AddBinderForm binder){
.....
}
Shall I have to put any json annotation on my BinderResponse class ?
I don't understand where am i wrong ?Anyone pleas guide me.
UPDATE :
public class BinderResponse extends WebApiResponseBase {
private String binderName;
private String binderAddress1;
public String getBinderName() {
return binderName;
}
public void setBinderName(String binderName) {
this.binderName = binderName;
}
public String getBinderAddress1() {
return binderAddress1;
}
public void setBinderAddress1(String binderAddress1) {
this.binderAddress1 = binderAddress1;
}
}
BinderView :
public class BinderView extends BaseView {
private String binderName;
private String binderAddress1;
public String getBinderName() {
return binderName;
}
public void setBinderName(String binderName) {
this.binderName = binderName;
}
public String getBinderAddress1() {
return binderAddress1;
}
public void setBinderAddress1(String binderAddress1) {
this.binderAddress1 = binderAddress1;
}
}
In console it prints data / BinderName :
In Loop
Data :: ada
In Loop
Data :: tya
New Update :
Here is BaseView :
#MappedSuperclass
public abstract class BaseView implements IEntity {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue
#Column(name="id")
private long id;
public long getId() {
return id;
}
public void setId(long id) {
if (this.id != 0 && this.id != id) {
throw new IllegalStateException(
"The ID must not be changed after it is set.");
}
this.id = id;
}
}
and In IEntity :
public interface IEntity extends Serializable {
long getId();
void setId(long id);
}
WebApiResponseBase :
public class WebApiResponseBase implements IWebApiResponse {
private String _uri;
#Override
public String getUri() {
return _uri == null ? "" : _uri;
}
#Override
public void setUri(String uri) {
_uri = uri;
}
}
Jackson, by default, serializes an object's whole inheritance hierarchy, ie. the parent class fields as well. In the case of
public class BinderResponse extends WebApiResponseBase {
it seems like
Could not write JSON: (was java.lang.NullPointerException) (through reference chain: java.util.ArrayList[0]->com.ngl.dto.outgoing.BinderResponse["valid"]); nested exception is com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: java.util.ArrayList[0]->com.ngl.dto.outgoing.BinderResponse["valid"])
Jackson tries to serialize a field called valid from a getter called isValid (which is a conventional bean property name). The getter method, however, seems to throw a NullPointerException for whatever reason.
If you want Jackson to ignore it, you can annotate the getter with #JsonIgnore or your class with #JsonIgnoreProperties and specify the property name, ie. valid.
In my case when I used #JsonIgnore the exception has been gone but the problem was it couldn't receive that value from API Request anymore and Spring ignored it (obviously because of #JsonIgnore) So I investigated about the issue and figured out that the problem was the getter and setter.
I had the Integer property while my getter was int. So when I changed the getter to Integer my problem solved and error's gone.
private Integer purchaseId;
#JsonIgnore
public int getPurchaseId() {
return purchaseId;
}
public void setPurchaseId(int purchaseId) {
this.purchaseId = purchaseId;
}
Changed to :
private Integer purchaseId;
public Integer getPurchaseId() {
return purchaseId;
}
public void setPurchaseId(Integer purchaseId) {
this.purchaseId = purchaseId;
}
#Column(name="createddate")
private Date createdDate;
#Transient
private String formatedCreatedDate;
public String getFormatedCreatedDate() {
DateFormat dateFormat = new SimpleDateFormat("dd/mm/yyyy");
return dateFormat.format(this.getCreatedDate());
}
It throws the same exception because here may be null by calling getCreatedDate() value come so it can't format null date so keep null check here like:
Solution
public String getFormatedCreatedDate() {
DateFormat dateFormat = new SimpleDateFormat("dd/mm/yyyy");
Date createDdate=this.getCreatedDate();
if(createDdate!=null){
return dateFormat.format(createDdate);
}
return "-";
}
I've run into some funky behavior with generics and I was wondering if someone could shed some light as to why this is happening. To start, I have a class Foo which has a field id. The hashCode method on Foo just returns the id. In another class I create a Map<Foo, Double> bar = new HashMap<Foo, Double().
Then, at a later part of the code the strangeness starts, and I am able to do the following (simplified here):
Long baz = new Long(1);
bar.get(baz);
So, my question is, Why doesn't the compiler and catch this and report it as an error?
EDIT: I made one mistake in my initial question in that get is the method that works, not put. I have posted the full code below.
Map<WebPage, Double> scoresForPhrase = new HashMap<WebPage, Double>();
// Now that we have a list of matching docs, we can calculate the
// Score of each word in the phrase for each document
for (String term: phrase.getWords()) {
TreeSet<Posting> wordPostings = wordMap.get(term);
for(Long doc: matchingDocs) {
if (docDenomScores.get(doc) == null) {
docDenomScores.put(doc, getDocTotal(doc));
}
// The set is of postings, which are compared by docId, so
// we need a temporary one to enable searching
Posting temp = new Posting(doc, new ArrayList<Integer>());
Posting wordPosting = wordPostings.ceiling(temp);
WebPage page = (WebPage) mWebpageDb
.getPageIdToWebPageTable().get(doc);
score = getTermScore(wordPosting, page,
wordPostings.size());
score = score * queryTermWeights.get(term);
Double curScore = scoresForPhrase.get(doc);
}
}
As for the Foo class, it is:
public class WebPage implements Serializable {
private static final long serialVersionUID = -4907557806357281837L;
private String mUrl;
private int mMaxTf;
private long mPageId;
private long mLastTimeUpdated;
private List<Long> mParentIds;
private long mContentLength;
private String mTitle;
private List<Long> mChildren;
private List<String> mAllUrls;
public WebPage(String url, long pageId, long lastTimeUpdated,
List<Long> parentIds, long contentLength, String title, List<Long> children,
List<String> allUrls) {
super();
this.mUrl = url;
this.mPageId = pageId;
this.mLastTimeUpdated = lastTimeUpdated;
this.mParentIds = parentIds;
this.mContentLength = contentLength;
this.mTitle = title;
this.mChildren = children;
this.mAllUrls = allUrls;
this.mMaxTf = 0;
}
public void setUrl(String mUrl) {
this.mUrl = mUrl;
}
public void setPageId(int mPageId) {
this.mPageId = mPageId;
}
public void setLastTimeUpdated(long mLastTimeUpdated) {
this.mLastTimeUpdated = mLastTimeUpdated;
}
public void setParentIds(List<Long> mParentId) {
this.mParentIds = mParentId;
}
public void setContentLength(long mContentLength) {
this.mContentLength = mContentLength;
}
public void setChildren(List<Long> mChildren) {
this.mChildren = mChildren;
}
public void setAllUrls(List<String> allUrls) {
this.mAllUrls = allUrls;
}
public void setMaxTf(int newTf) {
this.mMaxTf = newTf;
}
public String getUrl() {
return mUrl;
}
public long getPageId() {
return mPageId;
}
public long getLastTimeUpdated() {
return mLastTimeUpdated;
}
public List<Long> getParentIds() {
return mParentIds;
}
public long getContentLength() {
return mContentLength;
}
public List<Long> getChildren() {
return mChildren;
}
public String getTitle() {
return mTitle;
}
public List<String> getAllUrls() {
return mAllUrls;
}
public int getMaxTf() {
return mMaxTf;
}
#Override
public boolean equals(Object o) {
if (!(o instanceof WebPage)) {
return false;
} else {
return ((WebPage)o).mPageId == mPageId;
}
}
#Override
public int hashCode() {
return (int)mPageId;
}
public String toString() {
return mUrl;
}
}
So two things. First, remember that due to type-erasure there is no runtime checking of generic types. The Map<Foo, Double> simply becomes Map<Object, Object>.
Second, with regards to a compiler warning or error, you should get a warning or error if bar is declared of type Map<Foo, Double>. But if it is declared as Map, no warning or error. My guess is that bar is defined as Map bar.
UPDATE
The reason there is no error on get is that by definition get takes an Object not the generic type. It is one of the odd things about the interface.
Map.get
Your Map<Foo, Double> might have been casted to Map:
Map<Foo, Double> barOriginal = new HashMap<Foo, Double();
// ...
Map bar = barOriginal;
// ...
Long baz = new Long(1);
bar.put(baz, new Double(1));