AWS Elastic Beanstalk "Impaired services on all instances." - java

I have a spring service that I'm trying to load load into AWS Beanstalk. When i create the environment and upload my .war file it just stays stuck on degraded. When i look through the logs for errors i cannot see any errors. Also when try and connect to my url, for example http://something.us-east-1.elasticbeanstalk.com/, i get a 502 error. I've already looked at the documentation provided by amazon that states the red degraded message means that all/most of the requests to you page are failing. Any idea how i can find the issue? See the screenshot below for the Enhanced Health Overview.

So, it turned out that i was getting an error in my logs but i was not able to see them. I had to ignore all of the eb-something log files. I needed to be looking at the web-1.log. This file may be named different depending on your instance and your environment but this is where i could actually see my error.

people who are finding their actual log can look up for this section in their EB log file.
/var/log/web.stdout.log

I had the same problem, with "Impaired services on all instances."
As suggested above, I ssh'd into the elastic beanstalk instance and looked at /var/log/web.stdout.log.
As I found there, my problem was that I was following a django tutorial and created a django config file with the WSGIPath pointing to the project name that was in the tutorial, but my actual project had a different name.
I corrected the mistake, wiped the elastic beanstalk instance and set up the environment again from scratch.
No problems at all this time, everything turned green immediately.

Related

Configuring open telemetry for tracing service to service calls ONLY

I am experimenting with different instrumentation libraries but primarily spring-cloud-sleuth and open-telemetry ( OT) are the ones I liked the most. Spring-cloud-sleuth is simple but it will not work for a non-spring ( Jax-RS)project , so I diverted my attention to open telemetry.
I am able to export the metrics using OT, but there is just too much data which I do not need. Spring sleuth gave the perfect solution wherein it just traces the call across microservices and links all the spans with one traceId.
My question is - How to configure OT to get an output similar to spring-sleuth? I tried various configuration and few worked but still the information is huge.
My configuration
-Dotel.traces.exporter=zipkin -Dotel.instrumentation.[jdbc].enabled=false -Dotel.instrumentation.[methods].enabled=false -Dotel.instrumentation.[jdbc-datasource].enabled=false
However, this still gives me method calls and other data. Also, one big pain is am not able to SHUT DOWN metrics data.
gets error like below
ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter - Failed to export metrics. The request could not be executed. Full error message: Failed to connect to localhost/0:0:0:0:0:0:0:1:4317
Anyhelp will be appreciated -
There are 2 ways to configure the open telemetry agent(otel).
Environment variable
Java system property
you can either set
export OTEL_METRICS_EXPORTER=none
or
java -Dotel.metrics.exporter=none app.jar
Reference
https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/autoconfigure/README.md

Azure release pipeline not picking up variable from nestedStack.yml file

I'm configuring our Release pipeline so that Integration Tests are automatically run after pull requests are merged to master and deployed to Dev environment.
I'm currently getting a connection error, specifically java.net.UnknownHostException: and it looks like my one of my output variables from my nestedStack.yml code is not being imported/read properly:
my-repo/cloud-formation/nestedStack.yml
You can see there is a property there "ApiGatewayInvokeUrl" which is marked as an Output. It is used on Azure DevOps for the "Integration Testing" task in my "Deploy to Dev" stage. It is written as $(ApiGatewayInvokeUrl) as that's how variables on Azure DevOps are used.
This Deploy to Dev will "succeed", however when I further inspect the Integration Tests, I see none actually ran and there was a connection error immediately. I can see it is outputting the variable as $(ApiGatewayInvokeUrl) , so it looks like it just reads it as a String, and never substitutes it for the correct URL value:
I was going off the way another team set up there Integration Tests on a similar pipeline but I might have missed something. Do I need to define $(ApiGatewayInvokeUrl) somewhere in my codebase, or somewhere on Azure? Or am I missing something? I checked the other teams code and didn't see them define it anywhere else, that's why I am ultra confused.
Update: I went into AWS API Gateway and copied the invoke URL and hard-coded that into the Azure DevOps Maven (integration testing) goal path and now it's connecting. So it's 100% just not importing that variable for somer reason.
you need to define\create the variable to use it (unless its an automatic variable, and this one is definitely not an automatic variable). that variable isnt getting substituted because it doesnt exist (afaik).

Java Filter not working on production environment Websphere 8.5

I have a filter used for authorization. It's the first step entering application. Without it is not possible to do anything in the application.
I set a systemout on the very first line of doFilter method to monitor the behaviour, so comparing websphere server log files I'm sure that filter (same Ear deployed) on Test environment works and on Production environment not.
Maybe it's server configuration issue...?
I have access to read and modify WAS Console of test environment.
I have access only to read WAS Console of production environment.
So I can compare them, and maybe test some change on test environment to replicate the behavior and say to production administrator what exactly setup....
Any suggestion on which setting I can check (Was console, maybe in relation with Web.xml, etc...)?
Thx a lot for any suggestion.
EDIT
I was able to retrieve via FTP the EAR in InstalledApps of Production environment. I noted a file named "web_merged.xml" in which is missing the entry of the filter.
Maybe the problem is here? When is created and why? Why could be missing the entry there? How let WAS to create the right file (if the problem is there)?
The problem was the one on the EDIT in the question, webmerged.xml was wrong. We weren't able to understand why WAS generated this file in wrong way.
What I understand is generated during deploy and is a mix of web.xml of the applicatin with application server configuration.
That's a WebSphere issue, we guessed it has to do with application server cache. We asked to System Administrator to:
Uninstall the application
Stop the server in which the application was installed
Clear the cache
Restart the server
Reinstall application
From this point the file was generated in the right manner, as the application bahaviour.
Websphere version is 8.5

Exception "NoClassDefFoundError for javax/mail/Authenticator

I am trying to send mail in my web application. I am using the libraries javax.mail-1.4.4 and activation-1.1. My configuration is set up like described in the Tomcat Docs here http://tomcat.apache.org/tomcat-5.5-doc/jndi-resources-howto.html#JavaMail_Sessions
The web.xml file is configured properly according to this information. The context.xml where I store the settings is configured properly as well (or at least should be) with mail.*, auth, name, type info set properly.
When trying to send a mail this way, I get the following error:
Exception in thread "Thread-27" java.lang.NoClassDefFoundError: javax/mail/Authenticator
in this line:
session = (Session) envCtx.lookup("mail/Session");
I've looked through some solutions, like missing mail/activation libraries, checked up on those but none I could find were able to help me. Any idea what I could do to fix this or get in closer on the error?
"should be" is the operative phrase. I'd recommend that you believe Tomcat when it tells you it's missing things. You have not configured things properly until Tomcat says you have.
I'd recommend that you put them into the Tomcat /lib folder and see if that makes a difference.

Tomcat Deploying 2 Projects

I have a requirement that I need to use a single server machine to run 2 totally different applications.
The server machine has Apache Tomcat 6.0 installed. 1 of the applications is deployed on it and is running successfully for a long time. Initially, the clients used to connect to it by invoking the url "http://machine-name/1stProjectName/initialPage". I have also mentioned the "initialPage" as the Welcome Page in web.xml of this project. But somehow, later some ports or relating things were changed by the server maintenance team, and to connect to the application, now one has to use the url "http://machine-name" or "http://machine-name/initialPage", i.e. the machine name has started behaving like the project name folder because now if I invoke the url "http://machine-name/1stProjectName/initialPage", it gives 404 error saying that "/1stProjectName/1stProjectName/initialPage" resource not available.
Now, when I deploy the WAR of the second application into the webapps folder of Tomcat (just like I did for the first application), it unpacks it properly. But, the issue is how do I connect to this second project. If I use "http://machine-name", it takes me to the 1st application and if I use "http://machine-name/2ndProjectName/initialPage", it gives me an error that "/1stProjectName/2ndProjectName/initialPage" not available.
Kindly help or guide me in some direction.
Thanks in advance.
Your URLs should look like this if you did things properly:
Project 1: http://host:port/war-name-1
Project 2: http://host:port/war-name-2
That means no changes to content.xml to modify the path.
I would talk to your "server maintenance team" and let them know what you are trying to do (since they are the ones who made these changes in the first place). But it sounds to me that they just mapped a path to a specific location, that is your machine-name root is mapped to 1stProjectName.

Categories