We Keep Coding

2 Same HTTP Requests Give Different Results?

What differentiate these 2 requests that cause them to have different results/responses from the server although they should be the same ?
Request initiated by Chrome after a simple
click/navigation(successful, response code is 302)
I simply copied
that request as a curl and imported it to Postman and then postman
hanged
I did the same with Java - HttpUrlConnection(mimicking all the request headers and cookies like Chrome sent), but it hanged and waited forever. Is this simply because of the server logic that doesn't accept non-browser client ?
Here are the steps that I tried:
1. Visited this link: https://www.tokopedia.com/p/handphone-tablet/handphone
2. I opened the inspector and opened the Network - All tab
3. I clicked one of the products
4. I clicked the top request from the Network - All tab
5. I copied it as cURL bash
6. I imported it to Postman
7. I ran that request
8. Postman hanged

Actually the problem might even go deeper than what the other answers say.
So neither the User-Agent request header nor telnet might solve that problem (unless you initialize the TLS handshake also with telnet MANUALLY, but that is near impossible to complete).
TLS fingerprinting
If the connection is an SSL/TLS connection, the server could detect which algorithm is used to generate keys, and most applications have their specific signature / cipher.
So only by the TLS handshake alone you can tell Chrome from Postman or FireFox or Java. Java usually - unless a JVM implementation REALLY wants to go off-road - has the same signature across all platforms, using the same cipher/algorithm across all implementations.
I am sorry I cannot properly recall the name of this technique. The first project I know that published this is called something like "A3" or "S3". Salesforce published an article about JA3 analysis. They describe the technique and show a list of signatures and applications so you can guesstimate what app you're talking to, without the need to even decrypt the data: https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967
My Solution
I had that same problem too, wanted to scan the NVidia or AMD servers for graphics card availability. Did not work from Java, so after a lot of research, finding the project mentioned above, I simply used Selenium to control FireFox and that got the proper server responses and I achieved my goal this way.

The only way to be sure that the exact same data is sent is to manually send it yourself through something like telnet. I had a similar problem once- it turned out that the browser was sending the data in one big chunk, while my code was sending it line-by-line. No site should have this problem, but it's possible that it exists.

The server might be checking for User-Agent request header and will block traffic that does not originate from a browser. Try setting the header in curl or your Java Code to a value corresponding to (any) browser. I've encountered such behavior on some e-shops and commercial websites.

Prerender.io not caching pages - followed all steps as per documentation

We are trying to use pretender.io to our application which developed in AngularJS, Spring and Hibernate konnectnow.com which hosted at amazon server.
Here are the steps I followed:
Signup at prerender.io and got token: cFeRZcsv3JnAftreuhMO
Checked documentation and understood that I need to install middleware and decided to use Spring one.
In web.xml added pom added as mentioned https://github.com/greengerong/prerender-java
Added !# to the URL in all the pages.
Restarted tomcat server.
Logged into pretender.io with login details and found that nothing getting crawl.
For testing purpose the url konnectnow.com/#!/planpage changed to konnectnow.com/?_escaped_fragment_=/planpage
Nothing comes up, got error page isn’t working.
Checked Crawl Stats at pretender.io and found that as:
Status Code: 505, Cache Hit: Miss, Response Time(sec): 1.51sec, URL:
http://localhost:8080/#!/planpage
Not sure why it takes local host.
Can some one help me how to make this work.

We recommend using html5 push state instead of the #! in your URLs if possible. Html5 push state is better since nothing after a # is sent to the server, which can lead to issues for the crawlers that are checked by their user agent (Facebook, Twitter, etc).
You should set the forwardedURLHeader in order to have the Prerender Java middleware use a different host for your website instead of your proxy URL.
https://github.com/greengerong/prerender-java#forwardedurlheader
I also see that you posted your prerender token publicly so we regenerated your token to prevent someone else from using it. Please find your new token when you log into your Prerender.io account. I've also emailed you there.

Java application authentication using Active Directory

I am working on a third party Java application for which I need to authenticate its users using Active Directory.
This application is hosted on RHEL 6.5, and uses LDAP to authenticate with Windows Active Directory. The AD server has been set up, and is working fine with an earlier version of the application (which was configured to enable the integration).
For the newer version, the vendor has laid out some steps to modify/configure the application files to connect with the AD server, and which are expected to help us authenticate. After making required changes, we are only able to open the UI page, which does not accept the (correct) credentials, the page that helps us with application configuration is not accessible at all.
I am a newbie to this area, and as far as my understanding goes, I would just need to make the suggested changes, and not follow the steps presented in this document, as those are for authenticating the Linux users, and NOT the application's users. Could someone also please confirm the same.
Any help/guidance in this issue would be greatly appreciated.
Thanks very much!
Edit:
I am unable to provide details of the files as it is confidential property, but the changes have to be made in the server property files.
Kindly request more details if necessary, I will try my best to provide them as I can't disclose everything, citing confidentiality reasons.
Thank you very much.
Edit 2:
Have added more details with this edit, which are given below:
The application has one of its component as CAS, which is currently configured to use database as its authentication handler. When we enter the credentials - username: abcd, password: samplepswd, we are able to login successfully.
As the business requirement is that of authentication with Active Directory using LDAP, we have to modify the CAS properties file. As per instructions from the product vendor, we have changed the following properties to use ldap -
authenticationHandler.type=ldap
ldapSSLConfig.enabled=false
ldapContextSource.url=ldap://sample.ADserver.example.net:389
ldapContextSource.userDn=abcd
ldapContextSource.password=samplepswd
ldapAuthenticationHandler.filter=uid=%u
ldapAuthenticationHandler.searchBase=OU=DEF,OU=PQR,OU=XYZ,DC=ADserver,DC=example,DC=net
We also need to make changes in the casAuthConfig xml file for the following properties (as anonymous search is not supported):
1. anonymousReadOnly, value is set to false
2. java.naming.security.authentication, value is set to simple
There is provision to use ldap over SSL as well, but currently we are not using that.
However, if we do use SSL, additional changes have to be made to the following properties:
ldapSSLConfig.enabled=true
ldapSSLConfig.trustStorePath=/home/dir1/subdir1/subdir2/keystorename.keystore
ldapSSLConfig.trustStoreType=jceks
These are the only configuration changes done on our (client) side; and in fact the only changes done. Nothing has been added/modified on the server (AD server), except another user, but that has no impact on the existing setup.
After restarting cas to reflect the changes, we encounter the error of bad credentials, although the values entered are correct:
2015-09-16 12:12:30,558 INFO [com.emeter.cas.authentication.support.DelegatingAuthenticationHandler] - Authenticating credential using handler
com.emeter.cas.adaptors.ldappwd.BindLdapAuthenticationHandler
2015-09-16 12:12:30,558 DEBUG [com.emeter.cas.authentication.support.DelegatingAuthenticationHandler] - credentials.getUsername() = abcd
2015-09-16 12:12:30,672 INFO [com.emeter.cas.adaptors.ldappwd.BindLdapAuthenticationHandler] - Search for cn=abcd returned 0 results.
2015-09-16 12:12:30,672 INFO [org.jasig.cas.authentication.AuthenticationManagerImpl] - AuthenticationHandler:
com.emeter.cas.authentication.support.DelegatingAuthenticationHandler failed to authenticate the user which provided the following credentials:
[username: abcd]
2015-09-16 12:12:30,676 ERROR [org.jasig.cas.integration.restlet.TicketResource] - error.authentication.credentials.bad
org.jasig.cas.ticket.TicketCreationException: error.authentication.credentials.bad
at org.jasig.cas.CentralAuthenticationServiceImpl.createTicketGrantingTicket_aroundBody10(CentralAuthenticationServiceImpl.java:423)
Can anybody please help with this issue? Or possibly point in the right direction?
Any help would be greatly appreciated.
Thank you.

google app engine java basic authentication

I have started to write a webApp in Eclipse where ECMAScript6 on the client (testing mostly on chrome on Win7) communicates with the server side written in Java7 on Jetty+GAEplugin via xmlHttpRequest (1 get request, user edits data, 1 post requests ). if it gets more I will move to webSockets).
The client code does not know anything about logins and provides neither user name nor password because as I understand, the browser and the user handle that. On the server side I response with 401 if basic authentication is missing.
But when testing, the browser does not ask the user for any password and does not send a second request (FF the same effect). No errors. Chrome development console claims that my
response.addHeader("WWW-Authenticate","Basic realm=\"bus driver\"");
did not reach it.
A subset of my targeted user baser does not have a google account nor FB nor twitter. So OpenId nor OAuth are of no use for that subset. OAuth doc says: " The user signs in using her Google account, .." . Any way I try to search, directs me to ways to store user data, but I have problems with the communication between browser and servlet.
I have read that I should edit web.xml, but I cannot see how to plug my home-built authentication into that. Web.xml would be okay for me because there will be about 3 pages sharing the same (*3roles=9pages).
My software should be up to date. Oracle claims that I should stop using Java 7, but I cannot get local GAE to run on Java 8 ...
Just testing on FF and looking for stupid errors I may have made last night. FF claims: InvalidStateError: An aatempt was made to use an object that is not, or is no longer, usable. Hmm seems that there is at least something wrong with my XMLhttpRequest (it worked in chrome at least without authentication). I could test GET with authentication but without ECMAscript... Still my main question remains, where is the example with old school authentication on the new kid, the cloud?
Logcat is empty. I have seen the logs getting filled in TomCat or for Android. I may have to dig into this.

EWSHttpException: Connection not established

I'm using EWS Java API to connect Exchange server and retrieve information about mail, calendar appointment and task.
It's working well with a lot of user, except for one account.
I got the following error :
microsoft.exchange.webservices.data.EWSHttpException: Connection not established
at microsoft.exchange.webservices.data.HttpClientWebRequest.throwIfConnIsNull(HttpClientWebRequest.java:394)
at microsoft.exchange.webservices.data.HttpClientWebRequest.getResponseHeaders(HttpClientWebRequest.java:280)
at microsoft.exchange.webservices.data.ExchangeServiceBase.processHttpResponseHeaders(ExchangeServiceBase.java:1045)
at microsoft.exchange.webservices.data.SimpleServiceRequestBase.internalExecute(SimpleServiceRequestBase.java:58)
at microsoft.exchange.webservices.data.MultiResponseServiceRequest.execute(MultiResponseServiceRequest.java:144)
at microsoft.exchange.webservices.data.ExchangeService.bindToFolder(ExchangeService.java:350)
at microsoft.exchange.webservices.data.ExchangeService.bindToFolder(ExchangeService.java:374)
Here the code to establish the connection :
ExchangeService service = new ExchangeService();
ExchangeCredentials credentials = new WebCredentials(<user>, <password>);
service.setCredentials(credentials);
service.setUrl(new URI(url));
I suspect a specific account configuration for explaining this error but I'm unable to determine which parameter.

That exception is almost certainly due to a bug. I've seen it many, many times. The problem lies in the SimpleServiceRequest class. If there's an error when reading the response, it will close the response in a finally block in readResponse(). It will go back up to internalExecute(), where the catch block will try to process the headers...and it tries to read the response that has been closed. The closing won't null out the response, but it does null out some data in the response, which EWS tries to read as to display errors. Then you get another exception because the connection is null due to the response being closed earlier.
The solution is to either fix the bug yourself or enable tracing and look at the response to see what kind of error you're dealing with. Also, for good measure, make sure the Strings class is reading in the Strings.properties file or it'll throw a different exception when it can't find certain error messages.

After working with the debugger and Fiddler, one way I've seen this error coming in is from an HTTP 302 error (the server says the link has been moved permanently to an https: location instead of the almost identical http: location).
I'm going to guess that the Java EWS API is not using the Secure Sockets Layer correctly (and is attempting to send to an HTTP url instead of an HTTPS url).
EDIT
If you get past the 302 error, then you may very well have a problem with handling the SSL certificate properly later on. If you debug the API, you may be able to see one of those
"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
errors in on the stack in ServiceRequestBase.java. That means that a Truststore, somewhere cannot find a certificate, or it's not looking in the right place.
UPDATE
Check which NTLM flags are getting set in the EwsJCIFSNTLMScheme class. I've seen connections fail because these flags were getting set wrong.
Use something like Fiddler to automatically (and successfully) authenticate into your EWS instance, check and see what NTLM flags are getting set (by decoding the Authorization: Negotiate headers with Fiddler in the "Inspectors", "Auth" tab - it's a 32-bit hex number), and send those hex-valued flags into the Type1 and Type3 message constructors.

I have the same error in china.I think it's ews-java-api bug .so I check the github.com,I see the author Victor Boctor update the scrip.so I think maybe it can fix this bug.so try to compile the source code,and sure ，it fix this bug. ~_~ thanks for Victor Boctor