I'm totally new at Spring integration DSL stuff, what I'm trying to do is fetch all the RSS feeds from mongo, register them all into the flow context and have that process all the articles fetched from the feeds. Currently I have a for-each loop that calls a function containing the following:
IntegrationFlow newFlow = IntegrationFlows
.from(Feed.inboundAdapter(new URL(url), source), e -> e.id(org + "-" + source + "-feed").poller(Pollers.fixedDelay(poll).maxMessagesPerPoll(maxMessages).errorChannel("feedErrors")))
.enrichHeaders(h -> h.header("sourceId", sourceId))
.enrichHeaders(h -> h.header("sourceName", sourceName))
.enrichHeaders(h -> h.header("source", source))
.enrichHeaders(h -> h.header("categories", categories))
.enrichHeaders(h -> h.header("org", org))
.channel(MessageChannels.executor(taskExecutor))
.handle("enricher", "enhance")
.channel(news())
.get();
this.flowContext.registration(newFlow).id(org + "-" + source + "-flow").register();
It does so splendidly with the exception of exceptions, funnily enough. When a feed is currently unavailable, has been renamed or the article from the feed itself is malformed an exception is thrown and it goes to the "feedErrors" channel as indicated in the flow definition. That part looks as following:
#Component
public class RssFeedErrorHandler {
private final Logger logger = LoggerFactory.getLogger(RssFeedErrorHandler.class);
#Bean
public IntegrationFlow errorSender() {
return IntegrationFlows.from("feedErrors")
.handle("rssFeedErrorHandler", "handleError")
.get();
}
public void handleError(Message<MessagingException> message) {
logger.error(message.toString());
}
}
Exception examples:
AdviceMessage [payload=org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://news.google.com/rss/topics/CAAqIQgKIhtDQkFTRGdvSUwyMHZNRFZ4ZERBU0FtVnVLQUFQAQ?hl=en-GB&gl=GB&ceid=GB%3Aen, feedResource=null, metadataKey='google-news-politics.https://news.google.com/rss/topics/CAAqIQgKIhtDQkFTRGdvSUwyMHZNRFZ4ZERBU0FtVnVLQUFQAQ?hl=en-GB&gl=GB&ceid=GB%3Aen', lastTime=1613665200000}'; nested exception is java.io.FileNotFoundException: https://news.google.com/rss/topics/CAAqIQgKIhtDQkFTRGdvSUwyMHZNRFZ4ZERBU0FtVnVLQUFQAQ?hl=en-GB&gl=GB&ceid=GB%3Aen was successful, headers={id=101f3da9-42de-eb03-0da2-015952fc0a23, timestamp=1613667428995}, inputMessage=ErrorMessage [payload=org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://news.google.com/rss/topics/CAAqIQgKIhtDQkFTRGdvSUwyMHZNRFZ4ZERBU0FtVnVLQUFQAQ?hl=en-GB&gl=GB&ceid=GB%3Aen, feedResource=null, metadataKey='google-news-politics.https://news.google.com/rss/topics/CAAqIQgKIhtDQkFTRGdvSUwyMHZNRFZ4ZERBU0FtVnVLQUFQAQ?hl=en-GB&gl=GB&ceid=GB%3Aen', lastTime=1613665200000}'; nested exception is java.io.FileNotFoundException: https://news.google.com/rss/topics/CAAqIQgKIhtDQkFTRGdvSUwyMHZNRFZ4ZERBU0FtVnVLQUFQAQ?hl=en-GB&gl=GB&ceid=GB%3Aen, headers={id=8a470a5f-4027-371c-44fd-1250703077b0, timestamp=1613667428995}]]
AdviceMessage [payload=org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://www.startupbootcamp.org/feed/, feedResource=null, metadataKey='startupbootcamp-startups.https://www.startupbootcamp.org/feed/', lastTime=1613015630000}'; nested exception is java.net.ConnectException: Operation timed out (Connection timed out) was successful, headers={id=6b5a1a00-905a-68b6-128b-4cdd8eb06737, timestamp=1613664837150}, inputMessage=ErrorMessage [payload=org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://www.startupbootcamp.org/feed/, feedResource=null, metadataKey='startupbootcamp-startups.https://www.startupbootcamp.org/feed/', lastTime=1613015630000}'; nested exception is java.net.ConnectException: Operation timed out (Connection timed out), headers={id=70ef8ae7-1b75-9e38-e631-c88777741609, timestamp=1613664837150}]]
AdviceMessage [payload=org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://timesofindia.indiatimes.com/rssfeeds/1081479906.cms, feedResource=null, metadataKey='india-times-entertainment.https://timesofindia.indiatimes.com/rssfeeds/1081479906.cms', lastTime=1613715922000}'; nested exception is com.rometools.rome.io.ParsingFeedException: Invalid XML: Error on line 1: Character reference "�" is an invalid XML character. was successful, headers={id=fb936878-6450-d79a-851a-ca1ceeef8182, timestamp=1613705658735}, inputMessage=ErrorMessage [payload=org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://timesofindia.indiatimes.com/rssfeeds/1081479906.cms, feedResource=null, metadataKey='india-times-entertainment.https://timesofindia.indiatimes.com/rssfeeds/1081479906.cms', lastTime=1613715922000}'; nested exception is com.rometools.rome.io.ParsingFeedException: Invalid XML: Error on line 1: Character reference "�" is an invalid XML character., headers={id=24f2a11a-80a6-6c13-dbe1-9548747174aa, timestamp=1613705658734}]]
AdviceMessage [payload=org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://www.jeffbullas.com/feed/, feedResource=null, metadataKey='jeff-bullas-marketing.https://www.jeffbullas.com/feed/', lastTime=1613574000000}'; nested exception is java.io.IOException: Server returned HTTP response code: 500 for URL: https://www.jeffbullas.com/feed/ was successful, headers={id=e6998e35-afdc-b476-c2d9-2f4b0cc46c6b, timestamp=1613710313323}, inputMessage=ErrorMessage [payload=org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://www.jeffbullas.com/feed/, feedResource=null, metadataKey='jeff-bullas-marketing.https://www.jeffbullas.com/feed/', lastTime=1613574000000}'; nested exception is java.io.IOException: Server returned HTTP response code: 500 for URL: https://www.jeffbullas.com/feed/, headers={id=420f9b5d-1c58-cd0a-c4e2-5d640578f1c9, timestamp=1613710313322}]]
Here's two full stacks:
org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=https://timesofindia.indiatimes.com/rssfeeds/1081479906.cms, feedResource=null, metadataKey='india-times-entertainment.https://timesofindia.indiatimes.com/rssfeeds/1081479906.cms', lastTime=-1}'; nested exception is com.rometools.rome.io.ParsingFeedException: Invalid XML: Error on line 1: Character reference "�" is an invalid XML character.
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.getFeed(FeedEntryMessageSource.java:239)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.populateEntryList(FeedEntryMessageSource.java:202)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.doReceive(FeedEntryMessageSource.java:177)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.doReceive(FeedEntryMessageSource.java:58)
at org.springframework.integration.endpoint.AbstractMessageSource.receive(AbstractMessageSource.java:167)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:250)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:359)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.pollForMessage(AbstractPollingEndpoint.java:328)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$null$1(AbstractPollingEndpoint.java:275)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.lambda$execute$0(ErrorHandlingTaskExecutor.java:57)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:55)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$createPoller$2(AbstractPollingEndpoint.java:272)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.rometools.rome.io.ParsingFeedException: Invalid XML: Error on line 1: Character reference "�" is an invalid XML character.
at com.rometools.rome.io.WireFeedInput.build(WireFeedInput.java:236)
at com.rometools.rome.io.SyndFeedInput.build(SyndFeedInput.java:150)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.getFeed(FeedEntryMessageSource.java:226)
... 20 more
Caused by: org.jdom2.input.JDOMParseException: Error on line 1: Character reference "�" is an invalid XML character.
at org.jdom2.input.sax.SAXBuilderEngine.build(SAXBuilderEngine.java:232)
at org.jdom2.input.sax.SAXBuilderEngine.build(SAXBuilderEngine.java:303)
at org.jdom2.input.SAXBuilder.build(SAXBuilder.java:1196)
at com.rometools.rome.io.WireFeedInput.build(WireFeedInput.java:233)
... 22 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 15681; Character reference "�" is an invalid XML character.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
at org.apache.xerces.impl.XMLScanner.scanCharReferenceValue(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanCharReference(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at org.jdom2.input.sax.SAXBuilderEngine.build(SAXBuilderEngine.java:217)
... 25 more
org.springframework.messaging.MessagingException: Failed to retrieve feed for 'FeedEntryMessageSource{feedUrl=http://www.independent.co.uk/news/uk/politics/rss, feedResource=null, metadataKey='independent-politics.http://www.independent.co.uk/news/uk/politics/rss', lastTime=1613726122000}'; nested exception is com.rometools.rome.io.ParsingFeedException: Invalid XML: Error on line 1: DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true.
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.getFeed(FeedEntryMessageSource.java:239)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.populateEntryList(FeedEntryMessageSource.java:202)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.doReceive(FeedEntryMessageSource.java:177)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.doReceive(FeedEntryMessageSource.java:58)
at org.springframework.integration.endpoint.AbstractMessageSource.receive(AbstractMessageSource.java:167)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:250)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:359)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.pollForMessage(AbstractPollingEndpoint.java:328)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$null$1(AbstractPollingEndpoint.java:275)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.lambda$execute$0(ErrorHandlingTaskExecutor.java:57)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:55)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$createPoller$2(AbstractPollingEndpoint.java:272)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.rometools.rome.io.ParsingFeedException: Invalid XML: Error on line 1: DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true.
at com.rometools.rome.io.WireFeedInput.build(WireFeedInput.java:236)
at com.rometools.rome.io.SyndFeedInput.build(SyndFeedInput.java:150)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.getFeed(FeedEntryMessageSource.java:226)
... 20 more
Caused by: org.jdom2.input.JDOMParseException: Error on line 1: DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true.
at org.jdom2.input.sax.SAXBuilderEngine.build(SAXBuilderEngine.java:232)
at org.jdom2.input.sax.SAXBuilderEngine.build(SAXBuilderEngine.java:303)
at org.jdom2.input.SAXBuilder.build(SAXBuilder.java:1196)
at com.rometools.rome.io.WireFeedInput.build(WireFeedInput.java:233)
... 22 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 10; DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at org.jdom2.input.sax.SAXBuilderEngine.build(SAXBuilderEngine.java:217)
... 25 more
So the error is thrown, it's logged and life goes on, except for some reason that poll fetch seems to be saved for a retry. This wouldn't be an issue if it retried it once, but it just keeps retrying over and over, and when enough of errors pile up it just keeps retrying those errors and nothing new is ever fetched.
For example, 500 feeds are polled, 10 of those throw an exception where the feed has been either moved so it no longer exists, it has bad xml or something else might have happened. Now the poller is stuck polling those 10 feeds over and over again until it no longer fails (which never happens). I remember reading somewhere that if exception is thrown in handleError function then it would keep retrying, but if no exception is thrown there it should move on. That doesn't seem to be the case here. I've spend 3 days now mulling over this, trying different solutions but It always just chokes up after about 4-10 minutes, depending on how long it takes to fetch enough feeds with errors.
I would say that you try to talk about many different exceptions: not available, malformed, processing error etc. I think the best practice is to distinguish those exceptions and made an appropriate business decision for each of them: we definitely can't treat "not available" as processing error and must retry until it is available back.
For the processing one I would suggest to look into a ExpressionEvaluatingRequestHandlerAdvice on the service where your process entries of the feed: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#message-handler-advice-chain
For malformed source I'd suggest to take a look into a flow removal form the context. So, you won't try to poll a wrong source any more. You definitely can do that from the error flow on that feedErrors channel.
And so on for other errors you face with this scenario.
We probably may revise your 10 are of those throw an exception where the feeds have been removed for whatever reason in more details since the logic in the FeedEntryMessageSource is based on the updatedDate of the entry. So, if we couldn't convert it properly and produce, it is really going to be pulled again next time since the lastTime state has not been updated appropriately. But let's do that in the separate SO thread with much more details to investigate!
UPDATE
Some thoughts regarding your stack traces.
The exception like org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 15681; Character refer is not good and must be treated as fatal. You definitely can't parse that RSS source any more or at least until it comes back to normal. This kind of exception could be handled like a stop() for that Feed.inboundAdapter(). So, you catch an exception in the feedErrors channel flow and call the stop() of caused endpoint. Or even better just remove the whole dynamic flow for this RSS source.
The exception org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 10; DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true. is also not good and probably could be treated the same fatal way with subsequent stop() for the endpoint. Or you can investigate the xerces library how to allow that DTD declaration. Probably the option like syndFeedInput(SyndFeedInput) could also give you some hooks for the XML parsing. For example I see it has this option:
/**
* Since ROME 1.5.1 we fixed a security vulnerability by disallowing Doctype declarations by default.
* This change breaks the compatibility with at least RSS 0.91N because it requires a Doctype declaration.
* You are able to allow Doctype declarations again with this property. You should only activate it
* when the feeds that you process are absolutely trustful.
*
* #param allowDoctypes true when Doctype declarations should be allowed again, false otherwise
*/
public void setAllowDoctypes(boolean allowDoctypes) {
I am using embedded version of H2 1.4.191 with the following connection string:
jdbc:h2:<Path of database file>
I am facing two problems:
Sometimes my database size goes beyond 10GB and system stops working. I think the data stored in db should not take more than 5Mb. What is consuming memory?
Sometimes I am getting these exception :
org.h2.jdbc.JdbcSQLException: File corrupted while reading record: null. Possible solution: use the recovery tool [90030-191]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:168)
at org.h2.mvstore.db.MVTableEngine$Store.convertIllegalStateException(MVTableEngine.java:195)
at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:167)
at org.h2.mvstore.db.MVTableEngine.init(MVTableEngine.java:99)
at org.h2.engine.Database.getPageStore(Database.java:2460)
at org.h2.engine.Database.open(Database.java:692)
at org.h2.engine.Database.openDatabase(Database.java:270)
at org.h2.engine.Database.(Database.java:264)
at org.h2.engine.Engine.openSession(Engine.java:65)
at org.h2.engine.Engine.openSession(Engine.java:175)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:153)
at org.h2.engine.Engine.createSession(Engine.java:136)
at org.h2.engine.Engine.createSession(Engine.java:28)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:349)
at org.h2.jdbc.JdbcConnection.(JdbcConnection.java:107)
at org.h2.jdbc.JdbcConnection.(JdbcConnection.java:91)
at org.h2.Driver.connect(Driver.java:72)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at com.j256.ormlite.jdbc.JdbcConnectionSource.makeConnection(JdbcConnectionSource.java:252)
at com.j256.ormlite.jdbc.JdbcConnectionSource.getReadWriteConnection(JdbcConnectionSource.java:184)
at com.j256.ormlite.table.TableUtils.doCreateTable(TableUtils.java:440)
at com.j256.ormlite.table.TableUtils.createTable(TableUtils.java:220)
at com.j256.ormlite.table.TableUtils.createTableIfNotExists(TableUtils.java:61)
at com.limetray.pos.dbmanagers.DAO.init(DAO.java:75)
at com.limetray.pos.dbmanagers.DAO.createDB(DAO.java:172)
at com.limetray.pos.controllers.MainApp.appInitialization(MainApp.java:91)
at com.limetray.pos.controllers.MainApp.init(MainApp.java:80)
at com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:841)
at com.sun.javafx.application.LauncherImpl.lambda$launchApplication$155(LauncherImpl.java:182)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalStateException: File corrupted in chunk 1703, expected check value 1190, got 2040 [1.4.191/6]
at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:773)
at org.h2.mvstore.Page.read(Page.java:667)
at org.h2.mvstore.Page.read(Page.java:195)
at org.h2.mvstore.MVStore.readPage(MVStore.java:1939)
at org.h2.mvstore.MVMap.readPage(MVMap.java:736)
at org.h2.mvstore.Page.getChildPage(Page.java:217)
at org.h2.mvstore.Cursor.min(Cursor.java:129)
at org.h2.mvstore.Cursor.hasNext(Cursor.java:36)
at org.h2.mvstore.MVStore.loadChunkMeta(MVStore.java:689)
at org.h2.mvstore.MVStore.readStoreHeader(MVStore.java:670)
at org.h2.mvstore.MVStore.(MVStore.java:353)
at org.h2.mvstore.MVStore$Builder.open(MVStore.java:2888)
at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:154)
and
org.h2.jdbc.JdbcSQLException: General error: "java.lang.NullPointerException" [50000-178]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:344)
at org.h2.message.DbException.get(DbException.java:167)
at org.h2.message.DbException.convert(DbException.java:294)
at org.h2.engine.Database.openDatabase(Database.java:293)
at org.h2.engine.Database.<init>(Database.java:256)
at org.h2.engine.Engine.openSession(Engine.java:57)
at org.h2.engine.Engine.openSession(Engine.java:164)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:142)
at org.h2.engine.Engine.createSession(Engine.java:125)
at org.h2.server.TcpServerThread.run(TcpServerThread.java:150)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NullPointerException
at org.h2.mvstore.DataUtils.parseMap(DataUtils.java:630)
at org.h2.mvstore.MVStore.openMap(MVStore.java:411)
at org.h2.mvstore.db.TransactionStore.<init>(TransactionStore.java:96)
at org.h2.mvstore.db.MVTableEngine$Store.<init>(MVTableEngine.java:161)
at org.h2.mvstore.db.MVTableEngine.init(MVTableEngine.java:94)
at org.h2.engine.Database.getPageStore(Database.java:2355)
at org.h2.engine.Database.open(Database.java:659)
at org.h2.engine.Database.openDatabase(Database.java:262)
I think both are related to db corruption issue which is there in H2. How should I avoid this issue ? If not, Can I recover my data ?
I have a lotus notes hava agent that retrieves slide data from powerpoint files attached in lotus notes documents with the help of Apache POI. Sometimes it works and sometimes it doesn't, and if it doesn't work, here's the common error stack trace.
Java Console:
Start
blahblahblah.pptx
File has pptx extension
Reading embedded object...
Read existing presentation
org.apache.poi.POIXMLException: org.apache.poi.POIXMLException: java.lang.reflect.InvocationTargetException
at org.apache.poi.xslf.usermodel.XMLSlideShow.<init>(XMLSlideShow.java:91)
at org.apache.poi.xslf.usermodel.XMLSlideShow.<init>(XMLSlideShow.java:96)
at JavaAgent.NotesMain(JavaAgent.java:98)
at lotus.domino.AgentBase.runNotes(Unknown Source)
at lotus.domino.NotesThread.run(Unknown Source)
Caused by: org.apache.poi.POIXMLException: java.lang.reflect.InvocationTargetException
at org.apache.poi.xslf.usermodel.XSLFFactory.createDocumentPart(XSLFFactory.java:66)
at org.apache.poi.POIXMLDocumentPart.read(POIXMLDocumentPart.java:426)
at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:155)
at org.apache.poi.xslf.usermodel.XMLSlideShow.<init>(XMLSlideShow.java:89)
... 4 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:56)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:39)
at java.lang.reflect.Constructor.newInstance(Constructor.java:527)
at org.apache.poi.xslf.usermodel.XSLFFactory.createDocumentPart(XSLFFactory.java:64)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.xmlbeans.impl.store.Cur.createElementXobj(Cur.java:257)
at org.apache.xmlbeans.impl.store.Cur$CurLoadContext.startElement(Cur.java:2992)
at org.apache.xmlbeans.impl.store.Locale$SaxHandler.startElement(Locale.java:3198)
at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.reportStartTag(Piccolo.java:1082)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseAttributesNS(PiccoloLexer.java:1802)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseOpenTagNS(PiccoloLexer.java:1521)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseTagNS(PiccoloLexer.java:1362)
at org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.yylex(PiccoloLexer.java:4678)
at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.yylex(Piccolo.java:1290)
at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.yyparse(Piccolo.java:1400)
at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.parse(Piccolo.java:714)
at org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Locale.java:3439)
at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1270)
at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1257)
at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(SchemaTypeLoaderBase.java:345)
at org.openxmlformats.schemas.presentationml.x2006.main.SldDocument$Factory.parse(Unknown Source)
at org.apache.poi.xslf.usermodel.XSLFSlide.<init>(XSLFSlide.java:67)
... 12 more
Next Document
Finished scanning reports
Code Snippet:
if(extension.equalsIgnoreCase("pptx")) { // If file is 2007+ powerpoint file
System.out.println("File has pptx extension");
try {
System.out.println("Reading embedded object...");
IS = att.getInputStream(); //EmbeddedObject att = (EmbeddedObject) rtnav.getElement();
System.out.println("Read existing presentation");
XMLSlideShow pptxshow = new XMLSlideShow(IS);
System.out.println("Getting slides from file...");
As you can see, the error is thrown on the XMLSlideShow pptxshow = new XMLSlideShow(IS)
The attachment it read is only 1155 KB.
Promoting a comment to an answer...
The key bit in your exception is
Caused by: java.lang.OutOfMemoryError: Java heap space
This seems to be being caused by you using the default JVM heap size, and a very small default at that
Just bump up the maximum java heap size, and you should then be fine!
I am using hadoop1.1.1, while i am trying to execute wordcount example then i am getting eof exception while file in present on specified location.
Exception in thread "main" java.io.IOException: Call to
localhost/127.0.0.1:9100 failed on local exception:
java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1144)
at org.apache.hadoop.ipc.Client.call(Client.java:1112)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at $Proxy1.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:411)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:135)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:276)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:241)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:100)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1411)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1429)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:238)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:352)
at WordCountDriver.main(WordCountDriver.java:32)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:848)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:786)
Please help.
I am trying to create java files from wsdl using axis2 wsdl2java converter. The problem is that I keep getting this weird exception, which I cannot really explain.
At the moment I am using the example code in Wikipedia to test this.
Basically I call the program like this
wsdl2java -o "D:\Examples\SOAP" -wv 2 -uri "D:\Examples\test.wsdl"
And every time I get this exception
Exception in thread "main" org.apache.axis2.wsdl.codegen.CodeGenerationException
: Error parsing WSDL
at org.apache.axis2.wsdl.codegen.CodeGenerationEngine.<init>(CodeGenerationEngine.java:159)
at org.apache.axis2.wsdl.WSDL2Code.main(WSDL2Code.java:35)
at org.apache.axis2.wsdl.WSDL2Java.main(WSDL2Java.java:24)
Caused by: WSDLException: faultCode=ERROR: Exception occured while reading WSDL
2.0 doc: unknown protocol: d: org.apache.axis2.AxisFault: unknown protocol: d
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.readInTheWSDL
File(WSDL20ToAxisServiceBuilder.java:1172)
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.<init>(WSDL20
ToAxisServiceBuilder.java:153)
at org.apache.axis2.description.WSDL20ToAllAxisServicesBuilder.<init>(WS
DL20ToAllAxisServicesBuilder.java:53)
at org.apache.axis2.wsdl.codegen.CodeGenerationEngine.<init>(CodeGenerationEngine.java:102)
at org.apache.axis2.wsdl.WSDL2Code.main(WSDL2Code.java:35)
at org.apache.axis2.wsdl.WSDL2Java.main(WSDL2Java.java:24)
Caused by: java.net.MalformedURLException: unknown protocol: d
at java.net.URL.<init>(URL.java:574)
at java.net.URL.<init>(URL.java:464)
at java.net.URL.<init>(URL.java:413)
at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.readInTheWSDL
File(WSDL20ToAxisServiceBuilder.java:1168)
... 5 more
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.<init>(WSDL20
ToAxisServiceBuilder.java:155)
at org.apache.axis2.description.WSDL20ToAllAxisServicesBuilder.<init>(WS
DL20ToAllAxisServicesBuilder.java:53)
at org.apache.axis2.wsdl.codegen.CodeGenerationEngine.<init>(CodeGenerationEngine.java:102)
... 2 more
Does anyone know what might be causing this?
Edit
Exception 2:
Woden[Warning],0:0,Description-1001,The targetNamespace 'http://www.tmsws.com/wsdl20sample' is not dereferencable.
Exception in thread "main" org.apache.axis2.wsdl.codegen.CodeGenerationException
: Error parsing WSDL
at org.apache.axis2.wsdl.codegen.CodeGenerationEngine.<init>(CodeGenerationEngine.java:153)
at org.apache.axis2.wsdl.WSDL2Code.main(WSDL2Code.java:35)
at org.apache.axis2.wsdl.WSDL2Java.main(WSDL2Java.java:24)
Caused by: org.apache.axis2.AxisFault: Unable to find element {http://www.tmsws.com/wsdl20sample}request reffered to by operation Get
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.createAxisMessage(WSDL20ToAxisServiceBuilder.java:1112)
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.populateOperations(WSDL20ToAxisServiceBuilder.java:1053)
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.processInterface(WSDL20ToAxisServiceBuilder.java:956)
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.processService(WSDL20ToAxisServiceBuilder.java:384)
at org.apache.axis2.description.WSDL20ToAxisServiceBuilder.populateService(WSDL20ToAxisServiceBuilder.java:272)
at org.apache.axis2.description.WSDL20ToAllAxisServicesBuilder.populateAllServices(WSDL20ToAllAxisServicesBuilder.java:90)
at org.apache.axis2.wsdl.codegen.CodeGenerationEngine.<init>(CodeGenerationEngine.java:105)
... 2 more
wsdl2java -o "D:\Examples\SOAP" -wv 2 -uri "file:/D:/Examples/test.wsdl"
wsdl2java tries to read from the D:\... URL, you need to specify the URL (I think there is way to specify a local file instead of URL).