Hi I am facing problem in parsing the job status in Spring Batch [ I am new to spring-batch ]. In my ItemReader even though exception is occurring , status shows COMPLETED.
this is my job config.
#Bean
public Job customerJob(){
return jobBuilderFactory.get("custJob")
.listener(jobListner)
.start(custStep())
.build();
private Step custStep() {
return stepBuilderFactory.get("custStep1")
.<Customer,Customer>chunk(10)
.reader(customerItemReader)
.processor(customerItemProcessor)
.writer(customerItemWriter)
.allowStartIfComplete(true)
.build();
ItemReader class
#Override
public Customer read() throws Exception, UnexpectedInputException, ParseException,
NonTransientResourceException {
if(customerList != null && customerList.size() >=1){
Customer customer;
customer = customerList.remove(0);
System.out.println("## ID="+customer.getCustNumber());
return customer;
}
return null;
}
//Method to fetch list[from ItemStream interface]
#Override
public void open(ExecutionContext executionContext) throws ItemStreamException {
System.out.println("reading customer list");
try {
customerList = customerService.getCustomers();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Here getCustomers() method from line customerList = customerService.getCustomers() some time throw java.net.SocketException exception (if there is some problem in server). In this case I excepct that my job status should fail, but it shows "COMPLETED".
#Override
public void afterJob(JobExecution jobExecution) {
System.out.println("exit status ="+jobExecution.getExitStatus().getExitCode());
System.out.println(" status ="+jobExecution.getStatus().name());
}
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
//Line ommited
exit status =COMPLETED
status =COMPLETED
How to show status "FAIL" in such case.
Related
I don't know why is transaction is not completed.
Example)
public void transactionMethod() {
DefaultTransactionDefinition txDefinition = new DefaultTransactionDefinition();
txDefinition.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
TransactionStatus testTx = txManager.getTransaction(txDefinition);
try {
txManager.commit(testTx);
} catch (Exception ex) {
log.error("Catch!!!!!!!!!!!");
}
if (Objects.nonNull(testTx) && !testTx.isCompleted()) {
txManager.rollback(testTx);
}
}
#Test
public void test() {
SimpleTransactionStatus testTx = new SimpleTransactionStatus();
when(txManager.getTransaction(any())).thenReturn(testTx);
target.transactionMethod();
}
I think rollback is not implemented because commit ended in try{}.
However, testTx.isCompleted()is false.....
Could you tell me why testTx.isCompleted() is false?
I am new in netty I have a tcp client application developed with netty. When i use future get async response from server some response returning but future is not completing into timeout. TCPClient class like following;
public TcpClient {
public boolean connect(Host host) {
try {
Bootstrap clientBootstrap = new Bootstrap()
.group(group)
.channel(NioSocketChannel.class)
.option(ChannelOption.SO_KEEPALIVE,true)
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 50)
.remoteAddress(new InetSocketAddress(host.getIp(), host.getPort()))
.handler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) {
socketChannel.config().setRecvByteBufAllocator(new FixedRecvByteBufAllocator(2146));
FalconClientHandler falconClientHandler = new FalconClientHandler(host);
host.setFalconClientHandler(falconClientHandler);
socketChannel.pipeline().addLast(falconClientHandler);
}
});
channelFuture = clientBootstrap.connect().sync(); //BAŞARI İLE BAĞLANDI
channelFuture.channel().closeFuture().sync();
return host.isActive();
} catch (Exception e) {
log.info("Connection timed out --> " + e);
host.setActive(false);
return false;
} finally {
host.setActive(false);
}
}
public synchronized ResponseFuture send(long transactionId,String message) {
final Map<Long,ResponseFuture> responseFuture = new ConcurrentHashMap<>();
responseFuture.put(transactionId,new ResponseFuture());
if (!hostSelector.getUpHostList().isEmpty()) {
int hostCount = hostSelector.getUpHostList().size();
Host host;
host = hostSelector.getUpHostList().get(index.incrementAndGet() % hostCount);
if (host.isActive()) {
int headerLength = Integer.parseInt(message.substring(8, 12));
log.info("[{}] Host {} Tcp Request",message.substring(52, 52 + headerLength),host.getIp());
channelFuture.addListener((GenericFutureListener<ChannelFuture>) future -> {
log.info("[{}] Tcp request added to map",transactionId);
channelFuture.channel().pipeline().get(FalconClientHandler.class).setResponseFuture(responseFuture);
byte[] byteBuffer = message.getBytes();
channelFuture.channel().writeAndFlush(Unpooled.copiedBuffer(byteBuffer));
});
}
} else {
log.error("AYAKTA HOST YOK");
}
return responseFuture.get(transactionId);
}
}
Send method have transactionId and request message, When i send this message with transaction id response will return with this thransaction id. I am calling this send like following;
ResponseFuture responseFuture = falconClient.send(Long.valueOf(transactionId), finalMessage);
try {
Object obj = responseFuture.get(ddaTimeoutParam, TimeUnit.MILLISECONDS);
if(obj!=null) {
response = obj.toString();
ddaDelta = System.currentTimeMillis()-ddaRequestStartTime;
}
} catch (InterruptedException | ExecutionException | TimeoutException e) {
log.warn("[{}] DDA timeout. Timeout parameter: {}",transactionId,ddaTimeoutParam);
responseFuture.cancel(true);
response = "TIMEOUT";
ddaDelta = System.currentTimeMillis()-ddaRequestStartTime;
}
Response future is a basic Future implementation class. Put and get methods like that;
public class ResponseFuture implements Future<String> {
private volatile State state = State.WAITING;
ArrayBlockingQueue<String> blockingResponse = new ArrayBlockingQueue<String>(1);
private enum State {
WAITING,
DONE
}
#Override
public String get(long timeout, TimeUnit unit) throws InterruptedException,
ExecutionException, TimeoutException {
final String responseAfterWait = blockingResponse.poll(timeout, unit);
if (responseAfterWait == null) {
throw new TimeoutException();
}
return responseAfterWait;
}
public void set(String msg) {
if (state == State.DONE) {
return;
}
try {
blockingResponse.put(msg);
state = State.DONE;
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
My Handler class for receive server response message like following;
public class FalconClientHandler extends SimpleChannelInboundHandler<ByteBuf> {
private ChannelHandlerContext ctx;
private Map<Long,ResponseFuture> responseFuture;
public synchronized void setResponseFuture(Map<Long,ResponseFuture> responseFuture) {
log.info("{} ResponseFuture setted",responseFuture.keySet());
this.responseFuture = responseFuture;
}
#Override
public void channelRead0(ChannelHandlerContext channelHandlerContext, ByteBuf in) {
String input = in.toString(CharsetUtil.UTF_8);
String transactionKey = input.substring(52, 66).trim();
if(responseFuture.get(Long.valueOf(transactionKey))!=null)
responseFuture.get(Long.valueOf(transactionKey)).set(input);
else
log.info("[{}] Tcp Response map is empty",transactionKey);
}
}
When i run this code under high load like 30 transaction per second, tcp response returned from netty server but future get method received timeout.This situation not occuring every request for example %20 request is fail when 30 tps %50 request fail in 40 tps. What can be occur under load?
I am building a java REST application. Therefor I am using jboss on a wildfly 8 server.
The following code causes this exception: JBAS011469: Transaction is required to perform this operation (either use a transaction or extended persistence context)
#Path("/users")
#Stateless
public class UsersEndpoint {
#POST
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.APPLICATION_JSON)
public Response create(User user) {
try {
this.checkUsername(user.getUsername());
this.checkEmail(user.getEmail());
return Response.ok(userDAO.create(user)).build();;
} catch (IOException e) {
return Response.status(Response.Status.NOT_ACCEPTABLE)
.entity(e.getMessage())
.build();
} catch (Exception e) {
return Response.status(Response.Status.NOT_ACCEPTABLE)
.entity(e.getMessage())
.build();
}
}
#POST
#Path("username")
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.TEXT_HTML)
public Response username(String username) {
try {
this.checkUsername(username);
return Response.ok().build();
} catch (Exception e) {
return Response.status(Response.Status.NOT_ACCEPTABLE)
.entity(entity)
.build();
}
}
#POST
#Path("email")
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.TEXT_HTML)
public Response email(String email) {
try {
this.checkEmail(email);
return Response.ok().build();
} catch (Exception e) {
return Response.status(Response.Status.NOT_ACCEPTABLE)
.entity(this.validator.validateError(null, e))
.build();
}
}
private void checkUsername(String username) throws IOException {
try {
userDAO.get(username);
throw new IOException("Username is taken allready.");
}
catch (IOException e) { throw e; }
catch (Exception e) { System.out.println("Username can be used"); }
}
private void checkEmail(String email) throws IOException {
try {
userDAO.getByEmail(email);
throw new IOException("Email is taken allready.");
}
catch (IOException e) { throw e; }
catch (Exception e) { System.out.println("Email can be used"); }
}
}
The public Response email(String email) and public Response username(String username) functions are working fine. The Problem seems to be the call of both functions through the public Response create(User user) function:
...
try {
this.checkUsername(user.getUsername());
this.checkEmail(user.getEmail());
...
So now when I have a correct username but a duplicate email the exception JBAS011469: Transaction is required to perform this operation (either use a transaction or extended persistence context) is going to be thrown in the userDAO.getByEmail(email). When I have a duplicate username instead the IOException("Username is taken allready.") is going to be thrown as expected.
When I change the order of those functions to:
...
try {
this.checkEmail(user.getEmail());
this.checkUsername(user.getUsername());
...
the same problem occurs but in userDAO.get(username) and with a correct email and a duplicate username.
EDIT
When I am removing throw new IOException("Username is taken allready."); in private void checkUsername(String username) throws IOException { } the second DAO call is working fine. So the problem seems to be the Exception that is thrown. How can I solve that?
My UserDAO
public interface UserDAO {
public User create(User user) throws Exception;
public User get(String username) throws Exception;
public User getByEmail(String email) throws Exception;
}
My UserBean
#Stateless
#Remote(UserDAO.class)
public class UserBean implements UserDAO {
// Injected database connection:
#PersistenceContext private EntityManager em;
#Override
public User create(User user) throws Exception {
em.persist(user);
return user;
}
#Override
public User get(String username) throws Exception {
return em.createNamedQuery(User.QUERY_USERNAME, User.class)
.setParameter("userName", username)
.getSingleResult();
}
#Override
public User getByEmail(String email) throws Exception {
return em.createNamedQuery(User.QUERY_EMAIL, User.class)
.setParameter("email", email)
.getSingleResult();
}
}
My apologies for throwing this random subject, but I did not come up with a better name,
class ReportSenderRunnable implements Runnable {
private final LPLogCompressor compressor;
public ReportSenderRunnable(final LPLogCompressor compressor) {
this.compressor = compressor;
}
#Override
public void run() {
executeTasks();
}
private void executeTasks() {
try {
// compressor.compress();
reportStatus = ReportStatus.COMPRESSING;
System.out.println("compressing for 10 seconds");
Thread.sleep(10000);
} catch (final IllegalStateException e) {
logCompressionError(e.getMessage());
} /*catch (final IOException e) {
logCompressionError(e.getMessage());
}*/ catch (InterruptedException e) {
logCompressionError(e.getMessage());
}
try {
reportStatus = ReportStatus.SENDING;
System.out.println("sending for 10 seconds");
Thread.sleep(10000);
} catch (final InterruptedException e) {
reportStatus = ReportStatus.EXCEPTION_IN_SENDING;
}
try {
reportStatus = ReportStatus.SUBMITTING_REPORT;
System.out.println("submitting report for 10 seconds");
Thread.sleep(10000);
} catch (final InterruptedException e) {
reportStatus = ReportStatus.EXCEPTION_IN_SUBMITTING_REPORT;
}
System.out.println("Report Sender completed");
reportStatus = ReportStatus.DONE;
}
private void logCompressionError(final String cause) {
logError(ReportStatus.COMPRESSING, cause);
reportStatus = ReportStatus.EXCEPTION_IN_COMPRESSION;
}
private void logError(final ReportStatus status, final String cause) {
LOGGER.error("{} - {}", status, cause);
}
}
Ideally, statements like
System.out.println("sending for 10 seconds");
Thread.sleep(10000);
will be replaced by actual tasks, but for now assuming this is the case, and they way it runs is
private void submitJob() {
final ExecutorService executorService = Executors.newSingleThreadExecutor();
try {
final LPLogCompressor lpLogCompressor = getLpLogCompressor();
executorService.execute(getReportSenderRunnable(lpLogCompressor));
} catch (final IOException e) {
reportStatus = ReportStatus.EXCEPTION_IN_COMPRESSION;
LOGGER.debug("Error in starting compression: {}", e.getMessage());
}
System.out.println("started Report Sender Job");
}
My question was how to effectively test this code? The one I wrote is
#Test
public void testJobAllStages() throws InterruptedException, IOException {
final ReportSender reportSender = spy(new ReportSender());
doReturn(compressor).when(reportSender).getLpLogCompressor();
when(compressor.compress()).thenReturn("nothing");
reportSender.sendAndReturnStatus();
Thread.sleep(10);
assertEquals(ReportStatus.COMPRESSING, reportSender.getCurrentStatus());
Thread.sleep(10000);
assertEquals(ReportStatus.SENDING, reportSender.getCurrentStatus());
Thread.sleep(10000);
assertEquals(ReportStatus.SUBMITTING_REPORT, reportSender.getCurrentStatus());
}
This runs well for above code.
To me this is crappy for following reasons
Not all tasks would take same time in ideal cases
Testing with Thread.sleep will take too much time and also adds non-determinism.
Question
How do I test this effectively?
You could add a class with a method (e.g., TimedAssertion.waitForCallable) that accepts a Callable, which then uses an ExecutorService to execute that Callable every second until it returns true. If it doesn't return true in a specific period of time, it fails.
You would then call that class from your test like this:
boolean result;
result = new TimedAssertion().waitForCallable(() ->
reportSender.getCurrentStatus() == ReportStatus.COMPRESSING);
assertTrue(result);
result = new TimedAssertion().waitForCallable(() ->
reportSender.getCurrentStatus() == ReportStatus.SENDING);
assertTrue(result);
...etc. This way, you can easily wait for a particular state in your code to be true, without waiting too long -- and you can reuse this new class anywhere that you need this sort of assertion.
Based on #Boris the Spider comment, I made use of mocks and here is what my tests look like
#Mock
private ReportSenderRunnable reportSenderRunnable;
#Mock
private LPLogCompressor compressor;
#Before
public void setUp() throws Exception {
MockitoAnnotations.initMocks(this);
}
#Test(timeout = 1000)
public void testJobNoException() throws InterruptedException, IOException {
final ReportSender reportSender = spy(new ReportSender());
doReturn(compressor).when(reportSender).getLpLogCompressor();
when(compressor.compress()).thenReturn("nothing");
reportSender.sendAndReturnStatus();
Thread.sleep(10);
assertEquals("Job must be completed successfully", ReportStatus.DONE,
reportSender.getCurrentStatus());
}
#Test(timeout = 1000)
public void testJobWithIllegalStateException() throws Exception {
final ReportSender reportSender = spy(new ReportSender());
doReturn(compressor).when(reportSender).getLpLogCompressor();
doThrow(IllegalStateException.class).when(compressor).compress();
reportSender.sendAndReturnStatus();
Thread.sleep(10);
assertEquals("Job must failed during compression", ReportStatus.EXCEPTION_IN_COMPRESSION,
reportSender.getCurrentStatus());
}
I am having the Web-service method to start spring batch job.If any exception occurred in spring batch processing control is coming back till processor process method. But i need the controller to came back to web-service method there i have to catch and code to email that exception.
Web-service method:
public void processInputFiles() throws ServiceFault {
String[] springConfig = { CONTEXT_FILE_NAME };
ApplicationContext context = new ClassPathXmlApplicationContext(springConfig);
try {
setClientInfo();
JobLauncher jobLauncher = (JobLauncher) context.getBean(JOB_LAUNCHER);
Job job = (Job) context.getBean(REMITTANCE_JOB);
jobLauncher.run(job, new JobParameters());
}catch (Exception e) {
String errorMessage = "LockboxService exception:: Could not process Remittance(CSV) files";
final Message message = MessageFactory.createErrorMessage(MyService.class, errorMessage, e);
ErrorSenderFactory.getInstance().send(message, new Instruction[] { Instruction.ERROR_EMAIL });
}
Processor process method:
#Override
public Transmission process(InputDetail remrow) throws ServiceException {
try {
business logic here
}
catch(Exception e) {
throw new Exception("Unable to process row having the int number:");
}
}
Here is startJob which I use the to start the job in web application, Tye to throw specific exception
public boolean StartJob()
throws MyException{
try {
final JobParameters jobParameters = new JobParametersBuilder()
.addLong("time", System.nanoTime())
.addString("file", jobInputFolder.getAbsolutePath())
.toJobParameters();
final JobExecution execution = jobLauncher.run(job,
jobParameters);
final ExitStatus status = execution.getExitStatus();
if (ExitStatus.COMPLETED.getExitCode().equals(
status.getExitCode())) {
result = true;
} else {
final List<Throwable> exceptions = execution
.getAllFailureExceptions();
for (final Throwable throwable : exceptions) {
if (throwable instanceof MyException) {
throw (MyException) throwable;
}
if (throwable instanceof FlatFileParseException) {
Throwable rootException = throwable.getCause();
if (rootException instanceof IncorrectTokenCountException) {
throw new MyException(logMessage, errorCode);
}
if (rootException instanceof BindException) {
BindException bindException = (BindException) rootException;
final FieldError fieldError = bindException
.getFieldError();
final String field = fieldError.getField();
throw new MyException(logMessage, errorCode);
}
}
}
}
}
} catch (JobExecutionAlreadyRunningException ex) {
} catch (JobRestartException ex) {
} catch (JobInstanceAlreadyCompleteException ex) {
} catch (JobParametersInvalidException ex) {
} catch (IOException ex) {
} finally {
}
return result;
}
If Item processor is as below
#Override
public KPData process(InputDetail inputData) throws MyException {
try {
business logic here
}
catch(Exception e) {
throw new MyException("Some issue");
}
}