We Keep Coding

Difference between running in IntelliJ and running from a JAR file

I run the following simple example for a Jersey application on a Grizzly server:
#Path("example")
public class example {
#POST
public Response someMethod(#Context Request r) {
System.out.println("Received request at : " + LocalDateTime.now());
Response response = Response.ok().build();
return response;
}
}
public class GrizzlyServerMain {
public static HttpServer startServer(String BASE_URI) throws IOException {
// Scans for JAX-RS resources and providers in the specified code-package
final ResourceConfig resourceConfig = new PackagesResourceConfig("Application");
// create and start a new instance of grizzly http server exposing the Jersey application at BASE_URI
return GrizzlyServerFactory.createHttpServer(BASE_URI, resourceConfig);
}
public static void main(String[] args) throws IOException {
String BASE_URI = "http://0.0.0.0:8000";
// Base URI the Grizzly HTTP server will listen on
startServer(BASE_URI);
//noinspection ResultOfMethodCallIgnored
System.in.read();
}
}
When I run it from IntelliJ, everything works as expected.
Connected to the target VM, address: '127.0.0.1:61163', transport:
'socket' Dec 30, 2016 4:00:07 PM
com.sun.jersey.api.core.PackagesResourceConfig init INFO: Scanning for
root resource and provider classes in the packages: Application Dec
30, 2016 4:00:08 PM com.sun.jersey.api.core.ScanningResourceConfig
logClasses INFO: Root resource classes found: class
Application.Greetings class Application.example Dec 30, 2016 4:00:08
PM com.sun.jersey.api.core.ScanningResourceConfig init INFO: No
provider classes found. Dec 30, 2016 4:00:08 PM
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3
10/24/2016 03:43 PM' Dec 30, 2016 4:00:08 PM
org.glassfish.grizzly.http.server.NetworkListener start INFO: Started
listener bound to [0.0.0.0:8000] Dec 30, 2016 4:00:08 PM
org.glassfish.grizzly.http.server.HttpServer start INFO: [HttpServer]
Started
Received request at : 2016-12-30T16:00:13.097
Received request at : 2016-12-30T16:00:14.319
Received request at : 2016-12-30T16:00:15.583
Disconnected from the target VM, address:
'127.0.0.1:61163', transport: 'socket'
But when I create a JAR with all the dependencies, and as soon as I send the request from the client, I get:
WARNING: Exception during FilterChain execution
java.lang.IllegalStateException
at org.glassfish.grizzly.http.server.io.OutputBuffer.reset(OutputBuffer.java:217)
at org.glassfish.grizzly.http.server.Response.reset(Response.java:732)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:168)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:265)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:200)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:134)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:78)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:815)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:567)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:547)
at java.lang.Thread.run(Unknown Source)
Can you please tell me what could I be doing wrong?

The exception seems to have changed from IllegalStateException to IllegalArgumentException.
Have a look at Java Doc for ContainerProvider.
ContainerProvider is an SPI class and its implementation will be provided by the server you are using, in your case the Grizzly server.
As per Java Doc:
An implementation (a service-provider) identifies itself by placing a provider-configuration file (if not already present), "com.sun.jersey.spi.container.ContainerProvider" in the resource directory META-INF/services, and including the fully qualified service-provider-class of the implementation in the file..
So, I think you need to update "META-INF/services" to your jar and add the provider as "GrizzlyContainerProvider" (package qualified).
You can have a look at a similar post:
Grizzly and Jersey standalone jar

Kinesis worker error: Caught exception when initializing LeaseCoordinator

I am trying to execute the sample producer and consumer code on the Kinesis Streams website: http://docs.aws.amazon.com/streams/latest/dev/learning-kinesis-module-one-download.html
I've downloaded the source, and I am using Eclipse to run it. I've included the necessary jar files, so I would think that everything would be setup to run.
When I run the processor code that consumes the records from Kinesis, however, I get this error:
Aug 02, 2016 8:35:14 PM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker initialize SEVERE: Caught exception when initializing LeaseCoordinator
Does anyone think they could tell me what is causing this error?
EDIT: Here is the full stack trace from the error on Eclipse:
Aug 02, 2016 9:02:27 PM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker initialize
SEVERE: Caught exception when initializing LeaseCoordinator
com.amazonaws.services.kinesis.leases.exceptions.DependencyException: com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: User: arn:aws:sts::500238854089:assumed-role/NORD-NONPROD-a0121-Team/AEXM is not authorized to perform: dynamodb:CreateTable on resource: arn:aws:dynamodb:us-west-2:500238854089:table/amazon-kinesis-learning (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request ID: BGME094FRUAEK2KFCPQIAM5U8VVV4KQNSO5AEMVJF66Q9ASUAAJG)
at com.amazonaws.services.kinesis.leases.impl.LeaseManager.createLeaseTableIfNotExists(LeaseManager.java:124)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.KinesisClientLibLeaseCoordinator.initialize(KinesisClientLibLeaseCoordinator.java:172)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker.initialize(Worker.java:380)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker.run(Worker.java:324)
at com.amazonaws.services.kinesis.samples.stocktrades.processor.StockTradesProcessor.main(StockTradesProcessor.java:96)
Caused by: com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: User: arn:aws:sts::500238854089:assumed-role/NORD-NONPROD-a0121-Team/AEXM is not authorized to perform: dynamodb:CreateTable on resource: arn:aws:dynamodb:us-west-2:500238854089:table/amazon-kinesis-learning (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request ID: BGME094FRUAEK2KFCPQIAM5U8VVV4KQNSO5AEMVJF66Q9ASUAAJG)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1401)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:945)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:723)
at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:475)
at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:437)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:386)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:2074)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:2044)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.createTable(AmazonDynamoDBClient.java:899)
at com.amazonaws.services.kinesis.leases.impl.LeaseManager.createLeaseTableIfNotExists(LeaseManager.java:117)
... 4 more

Your stack trace is telling you exactly what the problem is:
User: arn:aws:sts::500238854089:assumed-role/NORD-NONPROD-a0121-Team/AEXM is not authorized to perform: dynamodb:CreateTable on resource: arn:aws:dynamodb:us-west-2:500238854089:table/amazon-kinesis-learning
Make sure you've provided credentials to the DynamoDBClient that has CreateTable permissions - LeaseCoordinator attempts to create the leasing table in Dynamo.

It is actually possible to configure logging for Scala Kinesis Enrich by running the jar file like this:
java -jar -Dorg.slf4j.simpleLogger.defaultLogLevel=debug snowplow-kinesis-enrich-0.5.0 --config enrich.conf --resolver resolver.json
This should print all debug messages from the Kinesis Client Library. (Watch out because the output will become very verbose.) Could you try rerunning with this change to logging? Hopefully that will provide more clues about what's going wrong.

Error using htmlunit

I'm using gargoylesoftware htmlunit in java to get webpages after their javascript has been run (as you can't do this with the inbuilt java libraries). I'm using this in a webscraper, so this code is being run many many times over without issue, however quite randomly I get this error every hour or so after running the program:
Jan 13, 2015 8:03:52 AM com.gargoylesoftware.htmlunit.WebConsole info
INFO: ThreadStats: couldn't get the catalog (undefined)
Jan 13, 2015 8:03:52 AM com.gargoylesoftware.htmlunit.WebConsole info
INFO: ThreadStats: couldn't get the catalog ()
The code this error is appearing on is:
final HtmlPage page = webClient.getPage(URIget.getSearchURL(ToSearch, board));
I'm pretty sure the error doesn't originate from my URI.getSearchURL method, as it doesn't do anything fancy, only stick a few strings together

Try putting these after creating a WebClient and it should prevent the message from showing
webClient.setCssErrorHandler(new SilentCssErrorHandler());
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.javascript.host.ActiveXObject").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.javascript.host.html.HTMLDocument").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.html.HtmlScript").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.javascript.host.WindowProxy").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache").setLevel(Level.OFF);

Problems when uploading large files to Amazon S3

I tried to use Amazon-SDK(Java) sample code S3TransferProgressSample.java to upload large files to Amazon-S3 storage (also posted here on AWS docs).
But when I am trying to upload 11 GB files, the upload is getting stuck at different points with the error message:
Unable to upload file to Amazon S3: Unable to upload part: Unable toexecute HTTP request: Unbuffered entity enclosing request can not be repeated " (attached screenshot).
It looks like that after IOException occurs SDK is not able to retry the request (see below).
Does anyone encounter this? What is the best-practice to resolve this? Any code is appreciated.
INFO: Received successful response: 200, AWS Request ID:
2B66E7669E24DA75<br> Jan 15, 2011 6:44:46 AM
com.amazonaws.http.HttpClient execute<br> INFO: Sending Request: PUT
s3.amazonaws.com /test_file_upload/autogenerated.txt Parameters:
(uploadId:
m9MqxzD484Ys1nifnX._IzJBGbCFIoT_zBg0xdd6kkZ4TAtmcG0lXQOE.LeiSEuqn6NjcosIQLXJeKzSnKllmw--, partNumber: 1494, )<br> Jan 15, 2011 6:45:10 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
**INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error**<br>
Jan 15, 2011 6:45:10 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
INFO: Retrying request<br> Jan 15, 2011 6:45:12 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:12 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
**INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error**<br>
Jan 15, 2011 6:45:12 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
INFO: Retrying request<br> Jan 15, 2011 6:45:13 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
**INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error**<br>
Jan 15, 2011 6:45:13 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
INFO: Retrying request<br> Jan 15, 2011 6:45:13 AM
com.amazonaws.http.HttpClient execute<br>
**WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated.**<br> Jan 15, 2011 6:45:14 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:14 AM com.amazonaws.http.HttpClient
execute<br> WARNING: Unable to execute HTTP request: Unbuffered entity
enclosing request can not be repeated.<br> Jan 15, 2011 6:45:14 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:15 AM com.amazonaws.http.HttpClient
execute<br> WARNING: Unable to execute HTTP request: Unbuffered entity
enclosing request can not be repeated.<br> Jan 15, 2011 6:45:16 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:16 AM
com.amazonaws.http.HttpClient
execute<br> WARNING: Unable to execute HTTP request: Unbuffered entity
enclosing request can not be repeated.<br> Jan 15, 2011 6:45:17 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:19 AM com.amazonaws.http.HttpClient
execute<br> WARNING: Unable to execute HTTP request: Unbuffered entity
enclosing request can not be repeated.<br> Jan 15, 2011 6:45:19 AM
com.amazonaws.http.HttpClient execute<br> ....<br> Jan 15, 2011
6:45:21 AM com.amazonaws.http.HttpClient handleResponse<br>
**INFO: Received successful response: 204, AWS Request ID: E794B8FCA4C3D007**<br> Jan 15, 2011 6:45:21 AM
com.amazonaws.http.HttpClient execute<br> ...<br> Jan 15, 2011 6:45:19
AM com.amazonaws.http.HttpClient execute<br> INFO: Sending Request:
DELETE s3.amazonaws.com /test_file_upload/autogenerated.txt
Parameters:<br> ...<br> Jan 15, 2011 6:47:01 AM
com.amazonaws.http.HttpClient handleErrorResponse<br> INFO: Received
error response: Status Code: 404, AWS Request ID: 0CE25DFE767CC595,
AWS Error Code: NoSuchUpload, AWS Error Message: The specified upload
does not exist. The upload ID may be invalid, or the upload may have
been aborted or completed.<br>

Try using the low level API.
This will give you far more control when things go wrong, as they are likely to do with an 11GB file.
Requests to and from S3 do fail from time to time. With the low level API, you'll be able to retry a part of the upload if it fails.
Refactoring the example in the Amazon docs a bit:
// Step 2: Upload parts.
long filePosition = 0;
for (int i = 1; filePosition < contentLength; i++) {
// Last part can be less than 5 MB. Adjust part size.
partSize = Math.min(partSize, (contentLength - filePosition));
// Create request to upload a part.
UploadPartRequest uploadRequest = new UploadPartRequest()
.withBucketName(existingBucketName).withKey(keyName)
.withUploadId(initResponse.getUploadId()).withPartNumber(i)
.withFileOffset(filePosition)
.withFile(file)
.withPartSize(partSize);
// repeat the upload until it succeeds.
boolean anotherPass;
do {
anotherPass = false; // assume everythings ok
try {
// Upload part and add response to our list.
partETags.add(s3Client.uploadPart(uploadRequest).getPartETag());
} catch (Exception e) {
anotherPass = true; // repeat
}
} while (anotherPass);
filePosition += partSize;
}
// Step 3: complete.
CompleteMultipartUploadRequest compRequest = new
CompleteMultipartUploadRequest(
existingBucketName,
keyName,
initResponse.getUploadId(),
partETags);
s3Client.completeMultipartUpload(compRequest);
Note: I am not a java developer so I could have messed things up syntactically, but hopefully this gets you going in the right direction. Also, you'll want to add in a 'retry counter' to prevent an endless loop if the upload repeatedly fails.

As a side note, 404 errors can be thrown if you try to do a multipart upload to a key that is already under a multipart upload.

I think you should try Multipart API supported by AWS.
Check this out : http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html

The answer of Geoff Appleford works for me.
However, I would add a && retryCount < MAX_RETRIES to the while loop control statement and increment of the retryCount on every exception caught inside the while.
Aviad

I wanted to add a comment to Geoff Appleford's answer but SO wouldn't allow me to. In general his answer to use low level API works fine but even if we do now have a do-while loop the way for loop is designed there is in-built retry logic. In his code snippet the file position increases only when there is a success otherwise you are uploading the same part again.

i m facing following error using first time eclipse

this error occurred what to do run the application??
Initializing AppEngine server
25/06/2010 9:16:57 AM com.google.apphosting.utils.jetty.JettyLogger info
INFO: Logging to JettyLogger(null) via com.google.apphosting.utils.jetty.JettyLogger
25/06/2010 9:16:59 AM com.google.apphosting.utils.config.AppEngineWebXmlReader readAppEngineWebXml
INFO: Successfully processed C:\Documents and Settings\kk\workspace\First\war\WEB-INF/appengine-web.xml
25/06/2010 9:16:59 AM com.google.apphosting.utils.config.AbstractConfigXmlReader readConfigXml
INFO: Successfully processed C:\Documents and Settings\kk\workspace\First\war\WEB-INF/web.xml
25/06/2010 2:47:14 PM com.google.appengine.tools.development.DevAppServerImpl start
INFO: The server is running at http://localhost:8888/
edit (1) ::
when i m running the program http://localhost:8888/first
(copied from a comment to my answer)
i face Error 403 FORBIDDEN what to do now?
in the firstservlet.java i write only the following code
package first;
import java.io.IOException;
import javax.servlet.http.*;
public class FirstServlet extends HttpServlet {
/**
*
*/
private static final long serialVersionUID = 1L;
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
resp.setContentType("text/plain");
resp.getWriter().println("Hello, world");
}
}
whats the wrong??

403 Forbidden is a HTTP status code returned by a web server when a user agent requests a resource that the server does not allow them to
(wikipedia)
The server is up and running, accepted you request but can't respond because you're not allowed to access some resource. So the problem is inside the Firstapplication, maybe this aplication trys to access some file system resources and the server does not have sufficient privileges.