I am currently getting timestamps from accelerometers, magnetometers, and gyroscopes and performing sensor fusion with GPS Location on an android device. I am getting the sensor timestamp using SensorEvent.timestamp and Location.getElapsedRealtimeNanos().
My code is as follows:
Sensor Timestamp
public void onSensorChanged( SensorEvent event ) {
if( event.sensor.getType() == Sensor.TYPE_ACCELEROMETER )
System.out.println( "Acc:" + event.timestamp );
}
GPS Timestamp
public void onLocationChanged( Location loc ) {
System.out.println( loc.getElapsedRealtimeNanos() );
}
My issue is that the timestamps are offset by some arbitrary amount. I know this because all the GPS values are clustered by some offset from the rest of the sensors. Sometimes this offset is in minutes, sometimes hours, sometimes leading and other times lagging. Why does this delay exist in my implementation and how do I fix it?
This is a quick plot of the clustering I was talking about. I sorted the timestamps and timestamps after the sharp disjoint are all timestamps that pertain to the GPS measurements.
In the logs, the data is output sequentially. Message types 1 and 4 pertain to sensor readings, while -1 pertains to GPS. As you can see, the timestamps are not monotonic. The rest of the GPS timestamps are offset from the sensors by a similar amount. Note that this datapoints are from another dataset.
I used the following code to output the time.
System.out.println( SystemClock.elapsedRealtimeNanos() );
After checking the system clock in the hooks, the GPS timestamp is consistent. However the sensor is clearly offset from the SystemClock. The first column is the SystemClock, second column is the timestamp from the respective event object, and the third timestamp is the message type (-1 for GPS, others are IMU sensors).
Things that I've looked into
I've also seen that GPS clock sync is about 10-15 seconds behind, but since I'm using the time from boot, it shouldn't be an issue.
I've looked into this SO question but I don't think it applies because the delay on that issue seems consistent (100 miliseconds) and the magnitude is small relative to what I'm experiencing.
As tempted as I am to use SystemClock.elapsedRealtime() in these event hooks, I know that there is a delay between when the sensors are measured and the events are called. I don't want to introduce any delay/uncertainty into my model.
After doing hours of digging, I have also found lots of android bugs that are a few years old and most are labelled obsolete. I am really stumped. Any light on the issue would be greatly appreciated.
The answer is simple, the SensorEvent.timestamp has an arbitrary zero reference:
It turns out after a bit of Googling (tip o' the hat to StackOverflow, as usual) that the timestamp one receives isn't based off of any particular 0-point defined in the Android OS or the API; it's an arbitrary per-sensor value intended to allow for different measurements from the same sensor to be compared, not for the measurements to be compared to other timestamped events. It's a known issue (bug concerning the documentation; bug concerning the behavior), but as of right now it's the way of the world for Android developers.
Source:
http://fixermark.blogspot.ca/2014/06/quirkiness-of-android-sensor-library.html
My solution is to estimate offsets by adding SystemClock.elapsedRealtimeNanos() into the log and estimating the delay/offset of each sensor.
Related
I have a requirement as follows
There are multiple devices producing data based on the device configuration. e.g., There are two devices producing data at their own intervals let’s say d1 producing for every 15 min and d2 producing for every 30 min
All this data will be sent to Kafka
I need to consume the data and perform calculations for each device which is based on the values produced for the current hour and the first value produced in the next hour. For e.g., If d1 is producing data for every 15min from 12:00 AM-1:00 AM then the calculation is based on the values produced for that hour and the first value produced from 1:00 AM-2:00 AM. If the value is not produced from 1:00AM-2:00 AM then I need to consider data from 12:00 AM-1:00 AM and save it data repository (Time series)
Like this there will be ‘n’ number of devices and each device has its own configuration. In the above scenario device d1 and d2 are producing data for every 1 hr. There might be other devices which will be producing data for every 3 hr, 6 hr.
Currently this requirement is done in Java. Since the devices are increasing so as the computations, I would like to know if Spark/Spark Streaming can be applied to this scenario?Any articles with respect to these kind of requirements can be shared so that it will be of great help.
If, and this is a big if, the computations are going to be device-wise, you can make use of topic partitions and scale the number of partitions with the number of devices. The messages are delivered in order per partition this is the most powerful idea that you need to understand.
However, some words of caution:
The number of topics may increase, if you want to decrease you may need to purge the topics and start again.
In order to ensure that the devices are uniformly distributed, you may consider assign a guid to each device.
If the calculations do not involve some sort of machine learning libraries and can be done in plain java, it may be a good idea to use plain old consumers (or Streams) for this, instead of abstracting them via Spark-Streaming. The lower the level the greater the flexibility.
You can check this. https://www.confluent.io/blog/how-choose-number-topics-partitions-kafka-cluster
Is there any way to get the results like in this link having wrong date and time set on device? Thank you.
There's a way1 ...
Suppose that you have deliberately set the clock 1 minute slow.
Create a file that contains the number 60,000. When an application wants to find the correct time, it calls System.getTimeMillis() to get what the system thinks the time is. Then reads the number from the file and adds that number to the result of System.getTimeMillis().
Clearly, this is NOT a good idea, but then neither is deliberately setting the system clock incorrectly. (For a start, if the clock is miss-set, then you will have difficulty syncing it with an external time source like an NTP server. That means that your system's clock will drift.)
The problem i am working on is the TOTP algorithm. The thing I was wondering is how would it work if the time on server and the one on the device are not the same.
I see. The answer is that TOTP cannot work if the two clocks are not synchronized to within a small multiple of the timestep.
1 - There's another way too. Write an application that can do some image processing on a picture of a clock to read the time. Then hook this up to your computer's video camera, and point the camera at a cuckoo clock hanging on your wall. Make sure you wind up the clock regularly. If you want the date as well, point a second video camera at your Dilbert desk calendar.
I am trying to conduct some extremely accurate data measurements. For this, I need to be able to get the current time in microseconds, accurate to 100 microseconds (Or more). I can't seem to be able to find any way on the Android Developer website. Device specific answers are acceptable (I have access to a Nexus 7, so any answers involving that would be awesome).
I had originally thought it possible to use the system sensors which give times accurate to the microsecond, however I have no idea how to set and/or tell if the sensors are accurate. Not to mention whether these event. - SensorManager
Is there any way to get the time in microseconds on an android device that is accurate to within 100microseconds?
you can use System.nanoTime(). according to doc
Returns the current timestamp of the most precise timer available on
the local system. This timestamp can only be used to measure an
elapsed period by comparing it against another timestamp. It cannot be
used as a very exact system time expression.
Returns
the current timestamp in nanoseconds.
From the java doc here you will get some extra explanation of it
My applcation description:
The application will generate the user location everywhere and everyime 24/7.
The other app which browse the locations of the user will show him the locations with 1-10 minute spaces(I havn't decided yet) and of course that the location record that will be printed will be the most accurate.
I have tried diffrent type of things but I can't get the most accurate location for a minute for example.
Many suggested to send to the requestLocationUpdate a minute as a parameter but then it will generate every minute a location but I want to get the most accurate location in that minute so I guess I will have too generate all of that minute locations and choose the most accurate.
I came across many errors such as getting a city level location which is pretty bad(You have been in xx:xx at new york city.. I'm not looking for that).
There are many considerations such as battery safe and accuracy.
I'm agree to compromise about the amount of locations (means print every 10 minute the user location instead of 1 minute).
Anyway I'm so confused, if someone got a plan (not code level) how to manage that system I would like to hear.
The GPS location provider should give you precision of meters if you are in open air. If you take samples every minute with requestLocationUpdate, that should be enough.
It is not possible to "get N samples in a given time interval and keep only the most accurate one", you should do that calculation keeping a buffer of positions if you need that. In any case, I don't think it is worth the effort in this kind of applications.
The battery life is going to be a problem if you want a mobile device to last more than 4-6 hours with the GPS + internet connection active.
I have been following android get duration from maps.google.com directions and I seem to be doing okay, but I have a question: does the duration returned in the JSON from google maps adjust for real time traffic?
From the definitions on the google developers documentation for directions api :
duration indicates the total duration of this leg
duration_in_traffic indicates the total duration of this leg, taking into account current traffic conditions
I would take this to mean that no, duration doesn't not adjust for real time traffic.
It's possible to determine the answer to the question empirically by getting the duration at different times of day. I think you'll find it is not real-time based.
The documentation does not make mention of real-time traffic. While that's not conclusive, accounting for real-time traffic conditions would be rather a coup and highly likely to be included.