Im writing a flask app using boilerpipe to extract content.Initially i wrote the boilerpipe extract as script to extract website content but when i try to integrate with my api JVM crashes when executing boilerpipe extractor . This is the error i get https://github.com/misja/python-boilerpipe/issues/17
i have also raised a issue in github
from boilerpipe.extract import Extractor
import unicodedata
class ExtractingContent:
#classmethod
def processingContent(self,sourceUrl,extractorType="DefaultExtractor"):
extractor = Extractor(extractor=extractorType, url=sourceUrl)
extractedText = extractor.getText()
if extractedText:
toNormalString = unicodedata.normalize('NFKD',extractedText).encode('ascii','ignore')
json_data = json.loads({"content": toNormalString, "url": sourceUrl , "status": "success", "publisher_id": "XXXXX", "content_count": str(len(toNormalString)) })
return json_data
else:
json_data = json.dumps({"response": {"message": "No data found", "url": sourceUrl , "status": "success", "content_count": "empty" }})
return json.loads(json_data)
This is the above script im trying to integrate in Flask api which use flask-restful,sqlachemy,psql . I also updated my java but that didn't fix the issue.Java version
java version "1.7.0_45"
javac 1.7.0_45
Any help would be appreciated
Thanks
(copy of what I wrote in https://github.com/misja/python-boilerpipe/issues/17)
OK, I've reproduced the bug : the thread that calls the JVM is not attached to it, therefore the calls to JVM internals fail.
The bug comes from boilerpipe (see below).
First, monkey patching : in the code you posted on stackoverflow, you just have to add the following code before the creation of the extractor :
class ExtractingContent:
#classmethod
def processingContent(self,sourceUrl,extractorType="DefaultExtractor"):
print "State=", jpype.isThreadAttachedToJVM()
if not jpype.isThreadAttachedToJVM():
print "Needs to attach..."
jpype.attachThreadToJVM()
print "Check Attached=", jpype.isThreadAttachedToJVM()
extractor = Extractor(extractor=extractorType, url=sourceUrl)
About boilerpipe: the check if threading.activeCount() > 1 in boilerpipe/extractor/__init__.py, line 50, is wrong.
The calling thread must always be attached to the JVM, even if there is only one.
Related
I am trying to figure out what is the best practice to use Mapbox on Android with tiles stored in a server.
I know the tiles are available because I can make them appear on the web.
I can use the default styles no problem mapBox.setStyle(Style.LIGHT) but when I try to change the map style using the server tiles.
I'm creating a JsonObject:
val mapJSONObject = JsonObject()
which is the equivalent of this:
{
"version": 8,
"sources": {
"tiles-name": {
"type": "raster",
"url": "https://myserver.com/map/c16c1u38213f0db8001b1c6e8cee5f7a/{z}/{x}/{y}?token=jfsdajfdsjkfjksh34232fsd",
"tileSize": 256
}
},
"layers": [{
"id": "ee-raster",
"type": "raster",
"source": "tiles-name",
"minzoom": 0,
"maxzoom": 22
}]
}
I've tried to use this JsonObject by either calling it by:
map.setStyle(Style.Builder().fromJson(mapJSONObject.toString()))
or by creating a .json file via this function:
#Throws(IOException::class)
fun save(context: Context, jsonString: String) {
val rootFolder = context.getExternalFilesDir(null)
val jsonFile = File(rootFolder, "file.json")
val writer = FileWriter(jsonFile)
writer.write(jsonString)
writer.close()
}
then using that .json file via:
map.setStyle(Style.Builder().fromJson("file.json"))
When triggering the change map using:
map.setStyle(Style.Builder().fromJson(mapJSONObject.toString()))
I get the error message:
D/Mbgl-HttpRequest: [HTTP] Request with response = 404: No additional
information
[Style]: Failed to load source tiles-name: HTTP status code 404
When triggering the change map using:
map.setStyle(Style.Builder().fromJson("file.json"))
I get the error message:
[ParseStyle]: Failed to parse style: Invalid value. at offset 1
I have trouble debugging this, even though the error messages seem clear enough.
I have been reading through the MapBox documentation, but can't seem to find a clear example using tiles from tiles on a server.
Can anyone identify what might be the issue or point me to an example using this method?
Doesn't have to be in Kotlin, Java would be equally useful.
I'd like to add some Java code snippets for Visual Studio Code, to speed up things that I commonly use, e.g. System.out.println() but the snippets are written in json, which I have no experience of. Can anybody help me with how to structure these Java snippets, and how you actually use them when programming?
This was my own attempt:
"Print to console":
{
"prefix": "System",
"body" : ["System.out.println("],
"description": "Print to the console"
}
Though I don't know whether I've written it wrong or whether I'm not accessing the snippet correctly.
From https://code.visualstudio.com/docs/editor/userdefinedsnippets
You can define your own snippets for specific languages. To open up a snippet file for editing, open User Snippets under File > Preferences (Code > Preferences on Mac) and select the language for which the snippets should appear.
After open up java.json using the step described above, you should see an example commented.
To fix your snippet, it should be:
"Print to Console": {
"prefix": "System",
"body": [
"System.out.println(\"$1\");"
],
"description": "Print to the console"
}
As you type System on vscode, you will see the IntelliSense to give you the suggestion.
I m new to RESTful Webservices in iOS and in Java. I read really a lot but i dont get definitiv answer how to SEND & RECEIVE a POST request in iOS & Handle that in Java.
My Situation is this:
I want to create a user on serverside. On Clientside i got for that 3 Objects that saves information (User, Usersdevice & Usersmembership). I read a lot about Objectmapping but I cant relate it with a practical example.
Seconde one is how to handle that POST on serverside with Java(Jersey) as RE.
Iknow that are two qeustions but I really need to know that.
I strongly recommend you to post a snippet of your code and clarify which restkit version uses. seems a bit outdated.
Also read the newest Object Mapping guide in order to leverage RestKit taking care if you want the KVC mapping or not. It could be tricky!!!
https://github.com/RestKit/RestKit/wiki/Object-mapping
First of all you have to determine your base URL:
RKObjectManager* manager = [RKObjectManager managerWithBaseURL:[NSURL URLWithString:#"http://www.yourURL.com]];
A method to download articles using RK object mapping:
- (void)loadArticles{
RKObjectMapping* articleMapping = [RKObjectMapping mappingForClass:[Article class]];
[articleMapping addAttributeMappingsFromDictionary:#{
#"title": #"title",
#"body": #"body",
#"author": #"author",
#"publication_date": #"publicationDate"
}];
RKResponseDescriptor *responseDescriptor = [RKResponseDescriptor responseDescriptorWithMapping:articleMapping pathPattern:nil keyPath:#"articles" statusCodes:RKStatusCodeIndexSetForClass(RKStatusCodeClassSuccessful)];
NSString * stringURL = #"/articles/";
[RKObjectManager.sharedManager getObjectsAtPath:stringURL parameters:nil success:^(RKObjectRequestOperation *operation, RKMappingResult *mappingResult){
RKLogInfo(#"Load collection of Articles: %#", mappingResult.array);
} failure:^(RKObjectRequestOperation *operation, NSError *error) {
RKLogError(#"Operation failed with error: %#", error);
}];
[objectRequestOperation start];
}
In order to map this JSON
{ "articles": [
{ "title": "RestKit Object Mapping Intro",
"body": "This article details how to use RestKit object mapping...",
"author": "Blake Watters",
"publication_date": "7/4/2011"
},
{ "title": "RestKit 1.0 Released",
"body": "RestKit 1.0 has been released to much fanfare across the galaxy...",
"author": "Blake Watters",
"publication_date": "9/4/2011"
}]
}
How can i parse the exported bookmarks file from Google Chrome and Mozilla Firefox in Java.Is there any libraries available to parse them directly and obtain the URLS in them.
Also sample codes for parsing them in Java are most welcomed .
In most cases, you don't really need to parse the HTML file. Chrome stores its bookmarks in a JSON file. It's a lot simpler to just read that file using a JSON parser.
The file you are interested in is located at (on Linux, anyway, Google around for other O/S):
/home/your_name/.config/google-chrome/Default/Bookmarks
JSON parsing is easy. Google around or start with How to parse JSON in Java.
If you want to visualize JSON data before you start digging through it, then also have a look at http://chris.photobooks.com/json/default.htm.
Per new comments posted , the solution would be to use JSOUP Open Source Program to do this.
JSOUP accepts only HTTP or HTTPS protocols so you might want to host the exported bookmark HTML on a Local Server like tomcat and obtain the DOM of it
http://yourip:<port>/<yourProject>/<bookmark.html>.
JSOUP is pretty self-explanatory.
Other simpler ways :
Chrome and Firefox bookmarks are stored as JSON like below.
Java way : I would suggest you use JSON to parse these. Make a reference Java Object based on the below structure.
or simply use UNIX Command prompt and do a
grep -i "url" <bookmark file path> | cut -d":" -f2
However if you still interested to do with Chrome APIs then please visit : http://developer.chrome.com/extensions/bookmarks.html
{
"checksum": "702d8e600a3d70beccfc78e82ca7caba",
"roots": {
"bookmark_bar": {
"children": [ {
"date_added": "12939920104154671",
"id": "3",
"name": "Development/Tutorials/Git/git-svn - KDE TechBase",
"type": "url",
"url": "http://techbase.kde.org/Development/Tutorials/Git/git-svn"
}, {
"date_added": "12939995405838705",
"id": "4",
"name": "QJson - Usage",
"type": "url",
"url": "http://qjson.sourceforge.net/usage.html"
I am a bit late to this question. But if it is still relevant: I needed to do the same (and also other bookmark sources: GitHub Stars, Netscape and Google Bookmarks as well) and build my own. You can look and take it from my repo: https://github.com/IvoLimmen/mystart.
If somebody is interested: Here's a scala snippet of how you could tackle parsing Chrome's bookmarks JSON file (not thoroughly tested though, just to get the idea):
import org.json4s.DefaultFormats
import org.json4s.native.JsonMethods
import org.junit.Test
class BookmarksImporterTest {
implicit val formats: DefaultFormats.type = DefaultFormats
def analyse(element: Node): List[Node] = {
element.children.flatMap(c => {
c.`type` match {
case Some("folder") => c.children.flatMap(r => analyse(r))
case Some("url") => List(c)
case _ => println("???"); List()
}
})
}
#Test
def test(): Unit = {
val source = scala.io.Source.fromFile("bookmarks.json")
val json = JsonMethods.parse(source.reader())
val bookmarks = json.extract[ChromeBookmarks]
val bms = bookmarks.roots.flatMap {
case (name, elements) => analyse(elements)
}
println("found " + bms.size + " entries")
}
}
case class ChromeBookmarks(checksum: String, roots: Map[String, Node], version: Int)
case class Node(
id: Option[String],
name: Option[String],
url: Option[String],
children: List[Node],
`date-added`: Option[Long],
`date-modified`: Option[Long],
`type`: Option[String]
)
EDIT: Note that this works perfectly in java 1.6 but fails in java 1.7.
I've been struggling to get the Couchbase 2.0 java API to work with views. It works perfectly for getting and putting keys into a bucket.
When I run the scala code below using Java 1.7, I get the following exception:
scala> ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [localhost/127.0.0.1:8092(closed)]
I've also tried setting the timeout in the connection builder to no avail.
import java.net.URI
import com.couchbase.client.CouchbaseClient
import scala.collection.JavaConversions._
val uris = List(URI.create("http://127.0.0.1:8091/pools"))
val client = new CouchbaseClient(uris, "test", "")
val view = client.asyncGetView("date", "dates")
However, the python code below works perfectly, connects to the view, and has the right output:
from couchbase.client import Couchbase
client = Couchbase("localhost:8091", "username", "password")
bucket = client["test"]
view = bucket.view("_design/date/_view/dates")
count = 0
for row in view:
count = count + 1
print(count)
Any ideas how to properly connect? I've tried to copy their examples exactly in my code. Unfortunately using python is not an option for this project.
we are aware of this issue (http://www.couchbase.com/issues/browse/JCBC-151).
It's not your fault or scalas, its just that our client currently has some problems to connect with java 7. If this is fixed, I'm sure your code will work as expected.