I'm working on a Java (11) project where we need to do some testing using Edge-Chromium (which is running on Linux via a docker container), so I've had to upgrade the version of Selenium we are using to 4.0.0-beta-4.
I've managed to get this bit working however when upgrading it seems that when I run any kind of testing now (locally or via the container), the logs are filled with GET/POST requests as if the browser itself is outputting all of its trace-level activity including what looks like memory dumps of the accessed pages (example below, imagine this x 500 and that's what the logs look like so completely illegible):
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 44 45 4c 45 54 41 20 2f 73 65 73 73 69 6f 6c 2f |DELETE /session/|
|00000010| 30 35 62 37 66 36 35 30 61 64 39 33 66 38 37 37 |05b234567d93f877|
|00000020| 65 65 39 31 31 31 30 33 39 37 63 31 33 30 65 64 |ee93110397c130ed|
|00000030| 20 48 54 54 50 2f 31 2e 31 0d 0a 55 73 65 72 2d | HTTP/1.1..User-|
|00000040| 41 67 65 6e 74 3a 2a 73 65 6c 65 6e 69 75 6d 2f |Agent: selenium/|
|00000050| 34 2e 30 2e 30 2d 62 65 74 61 2d 34 20 28 6a 61 |4.0.0-beta-4 (ja|
|00000060| 76 61 20 77 69 6e 64 6f 77 73 29 0d 0c 43 6f 6e |va windows)..Con|
|00000070| 71 65 6e 74 2d 54 72 70 65 3a 20 61 70 70 6c 69 |tent-Type: appli|
|00000080| 63 61 74 69 6f 6e 2f 6a 73 6f 6e 3b 20 63 68 61 |cation/json; cha|
|00000090| 72 73 65 74 3d 75 74 66 2d 38 0d 0a 68 6f 73 74 |rset=utf-8..host|
|000000a0| 3a 20 6c 6f 63 61 6c 68 6f 73 74 3b 33 33 28 38 |: localhost:3348|
|000000b0| 36 0d 0a 61 63 63 65 70 74 3a 20 2a 2f 2a 0d 0a |6..accept: */*..|
|000000c0| 0d 0a |.. |
+--------+-------------------------------------------------+----------------+
15:35:15.115 TRACE [id: 0x293801a8, L:/127.0.0.1:57141 - R:localhost/127.0.0.1:33486] FLUSH
15:35:15.136 TRACE [id: 0x293801a8, L:/127.0.0.1:57141 - R:localhost/127.0.0.1:33486] READ: 122B
This is actually impacting all the browsers I've used (Edge, Chrome and Firefox), they all output the same activity which makes me believe it's something to do with the Selenium upgrade itself and a package that comes with it rather than Edge specifically.
What I've tried so far:
Different flavours of Selenium 4 from 4.0.0-alpha-7 to 4.0.0-beta-4, all seem to have the same outcome.
Passing in logging preferences, these seem to make no difference no matter what values I put in:
// This is passing --silent in
System.setProperty(EdgeDriverService.EDGE_DRIVER_SILENT_OUTPUT_PROPERTY, "true");
System.setProperty(EdgeDriverService.EDGE_DRIVER_VERBOSE_LOG_PROPERTY, "false");
var loggingPrefs = new LoggingPreferences();
loggingPrefs.enable(LogType.PERFORMANCE, Level.WARNING);
loggingPrefs.enable(LogType.BROWSER, Level.WARNING);
loggingPrefs.enable(LogType.CLIENT, Level.WARNING);
loggingPrefs.enable(LogType.DRIVER, Level.WARNING);
loggingPrefs.enable(LogType.SERVER, Level.WARNING);
var options = new EdgeOptions();
options.setCapability(CapabilityType.LOGGING_PREFS, loggingPrefs);
options.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
options.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
var service = EdgeDriverService.createDefaultService();
if (headless) {
options.addArguments("--headless");
}
driver = new EdgeDriver(service, options);
Adding in lines for logback-test.xml to not report any logging for selenium:
<logger name="org.seleniumhq.selenium" level="OFF" />
<logger name="org.openqa.selenium" level="OFF" />
The steps outlined here although this issue seems slightly different to the one I'm experiencing.
The only thing that's given me any level of success is setting the following in logback-test.xml but it disables all the trace logging added in the project which isn't ideal:
<logger name="org.openqa.selenium" level="OFF" />
<root level="WARN">
<appender-ref ref="stdout" />
</root>
As this is specifically a logging issue (everything else is working otherwise) I feel like I'm missing an obvious logger or included project within Selenium 4 that I just need to turn off, but I've not been able to work out which logger it is - anyone have any ideas where I might be able to find this information or which logger I need to suppress? My guess is it relates to the appropriate browser driver (e.g. EdgeDriver) or WebDriver somehow but I would have thought those should be picked up by the turning off of org.openqa.selenium.
I discovered the answer to this myself so posting as an answer in the rare event anyone else experiences this issue. I've included how I worked it out as well in case other packages potentially do this in the future.
So to resolve this, I had to work out where the logs were coming from, so I added the following to my logback-test.xml file (I use Slf4j/Log4j in this project for reference) using this as a reference point:
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%logger %d{HH:mm:ss.SSS} %p %m %ex%n</pattern>
</encoder>
</appender>
This ended up flagging the following packages as being responsible after running the tests again and re-checking the logs:
io.netty
org.asynchttpclient.netty
So I then added the following to my logback-test.xml file (full xml included for reference):
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} %p %m %ex%n</pattern>
</encoder>
</appender>
<!-- Restrict logging for browser-level -->
<logger name="io.netty" level="WARN" />
<logger name="org.asynchttpclient.netty" level="WARN" />
<root level="TRACE">
<appender-ref ref="stdout" />
</root>
</configuration>
After doing this, the logs were no longer filled with the trace-level browser request logs as I encountered above. I'm not sure if they are part of the Selenium 4 package or if it's some other part of my specific project that's conflicted with the upgrade but this has restored the logging to how it was prior to the upgrade at any rate.
NOTE: The root level is defaulted to trace as I have trace-level logging throughout the project for diagnostic purposes when there is a test-based failure.
Related
I have a spring application with the following logback config file:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<logger name="PAPERTRAIL_LOGGER" level="error" additivity="false">
<appender name="PAPERTRAIL" class="ch.qos.logback.classic.net.SyslogAppender">
<syslogHost>[HOST]</syslogHost>
<port>[PORT]</port>
<facility>user</facility>
<suffixPattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] %clr(%5p) KOCH --- %ex{full}</suffixPattern>
</appender>
</logger>
</configuration>
But when I send a log message with the following code:
LoggerFactory.getLogger("PAPERTRAIL_LOGGER").error("Error: ", throwable);
Spring is sending two set of messages, one formatted by the suffixPattern provided:
Jun 13 10:45:04 DEVLINUSCUA03 DEVLINUSCUA03 [2022-06-13 14:45:04.320] ERROR KOCH --- org.postgresql.util.PSQLException: ERROR: relation "forcederror" does not exist Position: 15 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2365) [...]
and another broken in lines:
Jun 13 10:45:04 DEVLINUSCUA03 org.postgresql.util.PSQLException ERROR: relation "forcederror" does not exist Position: 15
Jun 13 10:45:04 DEVLINUSCUA03 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675)
Jun 13 10:45:04 DEVLINUSCUA03 at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2365)
Jun 13 10:45:04 DEVLINUSCUA03 at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:355)
Jun 13 10:45:04 DEVLINUSCUA03 at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:490)
[...]
How can I make it for spring to not send the second message to the syslog server?
Using your exact configuration (minus a postgres implementation) I get but a single log entry. I suspect that the postgres logger is doing this for you. Try disabling it and see what happens, then you can dive in to its configuration
<logger name="org.postgresql" level="OFF"/>`
In one of my camel route, i use camel's log eip as:
from("direct:someSourceWithBodyOfTypeMap")
.marshal().json(JsonLibrary.Jackson)
.to("direct:logHandler");
// on a separate route builder
from("direct:logForFilebeat")
.convertBodyTo(String.class, "UTF-8")
.log(LoggingLevel.INFO, "jsonLogger","${body}");
This route's purpose is to marshal a source of type Map to json and log the body which is later on picked up by a log aggregator. Im using logback for the logging. a portion of the logback-spring.xml looks like this:
<appender name="JSON_TX" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${BASE_LOG_PATH}/json.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${BASE_LOG_PATH}/json.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>%msg%n</pattern>
</encoder>
</appender>
<appender name="ASYNC_JSON_TX" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="JSON_TX" />
</appender>
<logger name="jsonLogger" level="INFO">
<appender-ref ref="ASYNC_FILEBEAT_TX" />
</logger>
In a normal flow, it gets properly logged as a json string like:
{"id":2456,"name":"foo","gender":"MALE"}
but there are instances that it gets logged as:
123 34 105 100 34 58 50 52 53 54 44 34 110 97 109 101 34 58 34 102 111 111 34 44 34 103 101 110 100 101 114 34 58 34 77 65 76 69 34 125
This decimals logs are generated when there are messages being processed by the route and the spring-boot up is triggered for shutdown. (I was able to replicate this locally by placing a breakpoint into processor before the source route, then shutting down spring-boot app while messages are routed to 'someSourceWithBodyOfTypeMap'.)
How do i enforce that the what gets appended to the log file should be in ASCII Text format not in decimal? And what triggers the logs to be in decimal in the first place?
Its not decimals its because your message body is marshalled as a stream of bytes. You can convert the message body to a String first by adding .convertBodyTo(String) after the marshal, or specify that the logger should convert to string via ${bodyAs(String)}.
The program uses logger (logback) to write a text. The text contains diacritical letters. Output is correct when the program runs on Eclipse (console). The project and all configuration based on UTF-8.
After create jar file and running it on Windows console (cmd), output has additional letters and lines (when the line contains diacritical letters).
Windows console configuration UTF-8 (chcp 65001) and font Lucida Console.
Why the additional lines appears?
/writeStdout/src/main/java/com/writeStdout/WriteStdout.java
package com.writeStdout;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class WriteStdout {
public static void main(String[] args) {
Logger logger = LoggerFactory.getLogger(WriteStdout.class);
logger.debug("This is a text without diacritical letters.");
logger.debug("This is a text with diacritical letters żółć żółć.");
logger.debug("This is a text without diacritical letters.");
logger.debug(
"This is a text with diacritical letters ślężańską źródłowość.");
logger.debug("This is a text without diacritical letters.");
}
}
Eclipse Console (proper output):
This is a text without diacritical letters.
This is a text with diacritical letters żółć żółć.
This is a text without diacritical letters.
This is a text with diacritical letters ślężańską źródłowość.
This is a text without diacritical letters.
Windows Console (additional lines and letters):
This is a text without diacritical letters.
This is a text with diacritical letters żółć żółć.
�łć.
This is a text without diacritical letters.
This is a text with diacritical letters ślężańską źródłowość.
owość.
This is a text without diacritical letters.
/writeStdout/src/main/resources/logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>UTF-8</charset>
<pattern>%msg%n
</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="STDOUT" />
</root>
</configuration>
/writeStdout/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.writeStdout</groupId>
<artifactId>writeStdout</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>writeStdout</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.source>1.8</maven.compiler.source>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
<configuration>
<archive>
<manifest>
<classpathPrefix>lib</classpathPrefix>
<addClasspath>true</addClasspath>
<mainClass>com.writeStdout.WriteStdout</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>
Now I have Java 1.8.0_144
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
Writing output to file
java -jar target\writeStdout-0.0.1-SNAPSHOT.jar > c:\logs\d.log
generate correct file
5468 6973 2069 7320 6120 7465 7874 2077
6974 686f 7574 2064 6961 6372 6974 6963
616c 206c 6574 7465 7273 2e0d 0a54 6869
7320 6973 2061 2074 6578 7420 7769 7468
2064 6961 6372 6974 6963 616c 206c 6574
7465 7273 20c5 bcc3 b3c5 82c4 8720 c5bc
c3b3 c582 c487 2e0d 0a54 6869 7320 6973
2061 2074 6578 7420 7769 7468 6f75 7420
6469 6163 7269 7469 6361 6c20 6c65 7474
6572 732e 0d0a 5468 6973 2069 7320 6120
7465 7874 2077 6974 6820 6469 6163 7269
7469 6361 6c20 6c65 7474 6572 7320 c59b
6cc4 99c5 bc61 c584 736b c485 20c5 ba72
c3b3 64c5 826f 776f c59b c487 2e0d 0a54
6869 7320 6973 2061 2074 6578 7420 7769
7468 6f75 7420 6469 6163 7269 7469 6361
6c20 6c65 7474 6572 732e 0d0a
This is a text without diacritical letters.
This is a text with diacritical letters żółć żółć.
This is a text without diacritical letters.
This is a text with diacritical letters ślężańską źródłowość.
This is a text without diacritical letters.
Command:
type c:\logs\d.log
shows correct content.
Writing text to file (logback level):
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="DEV_HOME" value="c:/logs" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>UTF-8</charset>
<pattern>%msg%n
</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>${DEV_HOME}/debug.log</file>
<append>true</append>
<immediateFlush>true</immediateFlush>
<encoder>
<charset>UTF-8</charset>
<pattern>%msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="FILE" />
</root>
</configuration>
generate correct file.
I've created a github project for it:
https://github.com/riskop/logback_diacritical.git
It works for me as expected:
C:\Users\riskop\Documents\test\logback_diacritical\target>ver
Microsoft Windows [Version 10.0.14393]
C:\Users\riskop\Documents\test\logback_diacritical\target>java -version
openjdk version "1.8.0_40"
OpenJDK Runtime Environment (build 1.8.0_40-b25)
OpenJDK Client VM (build 25.40-b25, mixed mode)
C:\Users\riskop\Documents\test\logback_diacritical\target>chcp
Active code page: 65001
C:\Users\riskop\Documents\test\logback_diacritical\target>java -jar
writeStdout-0.0.1-SNAPSHOT.jar
This is a text without diacritical letters.
This is a text with diacritical letters żółć żółć.
This is a text without diacritical letters.
This is a text with diacritical letters ślężańską źródłowość.
This is a text without diacritical letters.
What is your java / windows version?
Ps.: If I direct the output to file, I get this:
00000000 54 68 69 73 20 69 73 20 61 20 74 65 78 74 20 77 This is a text w...
00000010 69 74 68 6F 75 74 20 64 69 61 63 72 69 74 69 63 ithout diacritic...
00000020 61 6C 20 6C 65 74 74 65 72 73 2E 0D 0A 54 68 69 al letters...Thi...
00000030 73 20 69 73 20 61 20 74 65 78 74 20 77 69 74 68 s is a text with...
00000040 20 64 69 61 63 72 69 74 69 63 61 6C 20 6C 65 74 diacritical let...
00000050 74 65 72 73 20 C5 BC C3 B3 C5 82 C4 87 20 C5 BC ters ż ó ł ć ż....
00000060 C3 B3 C5 82 C4 87 2E 0D 0A 54 68 69 73 20 69 73 ó ł ć ...This is...
00000070 20 61 20 74 65 78 74 20 77 69 74 68 6F 75 74 20 a text without....
00000080 64 69 61 63 72 69 74 69 63 61 6C 20 6C 65 74 74 diacritical lett...
00000090 65 72 73 2E 0D 0A 54 68 69 73 20 69 73 20 61 20 ers...This is a....
000000A0 74 65 78 74 20 77 69 74 68 20 64 69 61 63 72 69 text with diacri...
000000B0 74 69 63 61 6C 20 6C 65 74 74 65 72 73 20 C5 9B tical letters ś....
000000C0 6C C4 99 C5 BC 61 C5 84 73 6B C4 85 20 C5 BA 72 lę ż ań ską ź r...
000000D0 C3 B3 64 C5 82 6F 77 6F C5 9B C4 87 2E 0D 0A 54 ó dł owoś ć ...T...
000000E0 68 69 73 20 69 73 20 61 20 74 65 78 74 20 77 69 his is a text wi...
000000F0 74 68 6F 75 74 20 64 69 61 63 72 69 74 69 63 61 thout diacritica...
00000100 6C 20 6C 65 74 74 65 72 73 2E 0D 0A l letters..........
What is your output in this form?
I wanted to install an ssl certification.
But I got:
#9: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 5C DE 33 E9 4A ED 08 31 64 1E 0E BE CC 6E AD AB \.3.J..1d....n..
0010: FD F0 56 C6 ..V.
]
]
Trust this certificate? [no]: y
Certificate was added to keystore
[Storing C:\Program Files\Java\jdk1.8.0_102\jre\lib\security]
keytool error: java.io.FileNotFoundException: C:\Program Files\Java\jdk1.8.0_102\jre\lib\security (Access is denied)
java.io.FileNotFoundException: C:\Program Files\Java\jdk1.8.0_102\jre\lib\security (Access is denied)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at java.io.FileOutputStream.<init>(FileOutputStream.java:101)
at sun.security.tools.keytool.Main.doCommands(Main.java:1144)
at sun.security.tools.keytool.Main.run(Main.java:343)
at sun.security.tools.keytool.Main.main(Main.java:336)
C:\Users\Houssem\Desktop\CERTIF_SSL>
I have to notice that I run the cmd as administrator
My windows is 8.1
UPDATE: I am also already Administrator
run as administrator (on linux using sudo command)!
Access is denied means you can't acces the file via your normal user.
When I run mvn site I get the error:
java.lang.NullPointerException
at org.codehaus.mojo.versions.ordering.MavenVersionComparator.innerGetSegmentCount(MavenVersionComparator.java:5
1)
at org.codehaus.mojo.versions.ordering.AbstractVersionComparator.getSegmentCount(AbstractVersionComparator.java:
27)
This appears to be a Maven bug, which according to the JIRA ticket has been resolved in version 1.2. I'm assuming this means version 1.2 of a particular plugin (probably the site plugin).
Does anyone know whether this fix has been released, and if so, how can I force Maven to use the version of the plugin that includes this fix?
Update
As suggested below, I added
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>versions-maven-plugin</artifactId>
<version>1.2</version>
</plugin>
to my pom.xml. But I now get the following error instead:
java.lang.NullPointerException
at org.codehaus.mojo.versions.PluginUpdatesDetails.isDependencyUpdateAvailable(PluginUpdatesDetails.java:68)
at org.codehaus.mojo.versions.PluginUpdatesRenderer.renderSummaryTotalsTable(PluginUpdatesRenderer.java:132)
at org.codehaus.mojo.versions.PluginUpdatesRenderer.renderBody(PluginUpdatesRenderer.java:71)
Thanks,
Don
It's the Codehaus Versions Maven Plugin.
Here's the config with the version you need:
<reporting>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>versions-maven-plugin</artifactId>
<version>1.2</version>
<reportSets>
...
</reportSets>
</plugin>
</plugins>
</reporting>
Edit:
For the new error, this source code line (68) suggests that a dependency is referenced without a version. Try looking for a plugin without a <version>.
63 public boolean isDependencyUpdateAvailable()
64 {
65 for ( Iterator i = dependencyVersions.values().iterator(); i.hasNext(); )
66 {
67 ArtifactVersions versions = (ArtifactVersions) i.next();
68 if ( versions.getAllUpdates( UpdateScope.ANY, includeSnapshots ).length > 0 )
69 {
70 return true;
71 }
72 }
73 return false;
74 }