We Keep Coding

How to calculate on UnOrdered message Arriving from source based on business condition

I have the following messages id and their occurrence time ,inserted by the ETL from the source in an unordered form.The business is as following
121 - start message
198 - end message
once we find end message calculate the total time from the start message.
Messages happened at the source
message id
occurence time
121
12:05
122
12:10
199
15:10
198
16:10
121
17:08
198
19:08
Messages inserted by ETL in the db table // etl cron run iteration 1
message id
occurence time
198
16:10
199
15:10
198
19:08
Messages inserted by ETL in the db table // etl cron run iteration 2
message id
occurence time
122
12:10
121
12:05
121
17:08
expected output shown in UI
121 -198 (12:05 - 16:10) calc total time
121 -198 (17:08 - 19:08) calc total time
The application runs after the etl run iteration 1 and find the records in unordered and needs to wait for the further records to calculate the total time.
how can we solve this problem. i was trying to solve by using variable sliding window algo or interval overlap algo? any idea ?

How to check time on all the nodes in hadoop cluster

I am running spark job on hadoop cluster, and the job is failing at few times with the exception :
exception : Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main() threw exception, begin > end in range (begin, end): (1494159709088, 1494159706071)
the job ran successfully on the rerun.
After searching on google, It might be Clock skew between the Oozie server host and launcher host.
Is there a way i can check if there is clock skew ? or how can i check the time on all the nodes whether they are in sync or not.
Thanks
ntptime command output :
ntp_gettime() returns code 0 (OK)
time dcb9b19b.a2328f64 Sun, May 7 2017 14:45:47.633, (.633584090),
maximum error 434990 us, estimated error 815 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
modes 0x0 (),
offset 176.871 us, frequency -25.666 ppm, interval 1 s,
maximum error 434990 us, estimated error 815 us,
status 0x2001 (PLL,NANO),
time constant 10, precision 0.001 us, tolerance 500 ppm,
ntpstat command output :
synchronised to NTP server (174.68.168.57) at stratum 3
time correct to within 77 ms
polling server every 1024 s

Reading Card Number on card given you have UID of IClass Card

I have HID Class Reader USB Device 5427 CK . I have successfully read ATR and UID of the card . The problem is there is a number printed on card .
i.e 2x01966 21093771-1
The Output i am getting while reading the card with java API .
PC/SC card in HID OMNIKEY 5427 CK 0, protocol T=1, state OK
ATR = 0x3B 8F 80 01 80 4F 0C A0 00 00 03 06 0A 00 1C 00 00 00 00 7E
Card UID = 0xB6 A9 0E FB FF 12 E0
Card type: PicoPass 16KS (8x2)
How can get the same number from UID

See the OMNIKEY Contactless Smart Card Readers Developer Guide available here. It tells how to communicate with iClass cards in chapters 9 and 10.
The command Select Page with P2=0x04 (request for 8-byte card serial number) might work (page 38).
EDIT>
Read the above-mentioned manual first.
Download the "Synchronous API for OMNIKEY Contactless Smart Card readers" available here (as written in the chapter 6).
Install the downloaded package and study the provided example iClassExplorer under "C:\Program Files\HID Global\Sync-API".
If you insist on calling it from java, you might find JNA useful.
The javax.smartcardio way probably won't work as you need to use the SCardCLICCTransmit function from the scardsyn.dll (chapter 9.1).

SCardCLICCTransmit() is a propriatary API call residing in OMNIKEY synchronous API DLL.
Use native CCID drivers with this reader instead and access UID via FFCA0000. This should work perfectly fine without additional wrappers. This APDU is availav=ble for all PC/SC part 3 compatible contactless smart card readers.

HTTPClient 4.x - first http request significantly slower

I changed my implementation using a newer version of Apache HTTP Client (3.x before, now 4.x)
My problem is, that the first HTTP request of a series of 4 different GET requests takes significantly
longer than the following ones. This problem is reproducable, if a pause of 60 seconds is done between each request-series. below there are my results of a simple benchmark. server code stays the same and also the requests are always the same.
I think the problem is something inside the client is destroyed after not used and re-created
It would be help if somebody has any idea what I could configure to avoid this poor performance.
Apache HTTP Client 3.x
REQUEST A duration in ms >> 25
REQUEST B duration in ms >> 20
REQUEST C duration in ms >> 15
REQUEST D duration in ms >> 15
----->>> 60 sec pause
REQUEST A duration in ms >> 20
REQUEST B duration in ms >> 15
REQUEST C duration in ms >> 15
REQUEST D duration in ms >> 15
Apache HTTP Client 4.2.6
REQUEST A duration in ms >> 4659
REQUEST B duration in ms >> 25
REQUEST C duration in ms >> 20
REQUEST D duration in ms >> 20
----->>> 60 sec pause
REQUEST A duration in ms >> 4510
REQUEST B duration in ms >> 20
REQUEST C duration in ms >> 15
REQUEST D duration in ms >> 10
Thank you very much in advance
Shane

Thanking you very much. To be more exact, i am using HTTP Client 4.2.6 inside the Resteasy 3.0.9 framework.
Some example would be this code:
ProxyFactory.create(ITestService.class, baseUrl, new ApacheHttpClient4Executor(httpClient));
The problem was (means it is reproducable) that i was using a deprecated contrutor of the Scheme class fore registering a new scheme to apache "Scheme Registry"
SchemeRegistry registry = new SchemeRegistry();
WRONG -> using deprecated Scheme constructor
registry.register(new Scheme("http",PlainSocketFactory.getSocketFactory(), usedPort));
correct way
registry.register(new Scheme("http",usedPort,PlainSocketFactory.getSocketFactory()));

What does sub error code 568 mean for Ldap Error 49 with Active Directory

I am writing some Java code that authenticates to Active Directory using SASL GSSAPI. Mostly this code is working fine but for one user I am getting the response:
javax.naming.AuthenticationException: [LDAP: error code 49 - 8
0090304: LdapErr: DSID-0C0904D1, comment: AcceptSecurityContext error, data 568,
v1772 ]
I know that 49 means this is an authentication failure, and that the relevant sub code is 568, but I am only aware of the following meanings for that data:
525 - user not found
52e - invalid credentials
530 - not permitted to logon at this time
532 - password expired
533 - account disabled
701 - account expired
773 - user must reset password
So far I am unable to find an authorative source of these error codes from Microsoft (this list is pieced together from forum posts) and I can't find anything for that 568 error.
Does anyone know what it means?
EDIT: It looks like the source of this list comes from this documentation from IBM

This and this list contain error codes that seem to correspond to the above numbers, viz.
ERROR_NO_SUCH_USER 1317 (0x525) The specified account does not exist.
ERROR_LOGON_FAILURE 1326 (0x52E) Logon failure: unknown user name or bad password.
ERROR_INVALID_LOGON_HOURS 1328 (0x530) Logon failure: account logon time restriction violation.
ERROR_PASSWORD_EXPIRED 1330 (0x532) Logon failure: the specified account password has expired.
ERROR_ACCOUNT_DISABLED 1331 (0x533) Logon failure: account currently disabled.
ERROR_ACCOUNT_EXPIRED 1793 (0x701) The user's account has expired.
ERROR_PASSWORD_MUST_CHANGE 1907 (0x773) The user's password must be changed before logging on the first time.
From this list it appears that this error code means:
ERROR_TOO_MANY_CONTEXT_IDS 1384 (0x568) During a logon attempt, the user's security context accumulated too many security IDs.
It turns out that this account has 2000 group memberships which are overrunning an internal Active Directory limit. You may only have 1015 or so group memberships otherwise login will fail.
More information is available on this error at: http://go.microsoft.com/fwlink/?LinkId=146571.