We Keep Coding

Text normalization in python using Normalizer.Form.NFKD

A field in the table is normalized using Java as shown below,
String name = customerRecord.getName().trim();
name = name.replaceAll("œ", "oe");
name = name.replaceAll("æ", "ae");
name = Normalizer.normalize(name, Normalizer.Form.NFKD).replaceAll("[^\\p{ASCII}]", "");
name = name.toLowerCase();
Now I'm trying to query the same db using Python. How do I do Normalizer.normalize(name, Normalizer.Form.NFKD) in Python so that it is compatible with the way it is written to?

An almost complete translation of the above Java code to Python would be like as follows,
import unicodedata
ASCII_REPLACEMENTS = {
'œ': 'oe',
'æ': 'ae'
}
text = ''.join([ASCII_REPLACEMENTS.get(c, c) for c in search_term])
ascii_term = (
unicodedata.normalize('NFKD', text).
encode('ascii', errors='ignore').decode()
)
return ascii_term.lower()
ASCII_REPLACEMENTS should be amended with what ever characters that wont get translated correctly by unicodedata.normalize compared to Java's Normalizer.normalize(name, Normalizer.Form.NFKD). This way we can ensure the compatibility between the two.

How to convert version one working url(endpoint) to java query using version one java REST client

I need to convert below working REST endpoint to java query
/rest-1.v1/Data/Timebox?Where=Schedule.ScheduledScopes.Name="Sample: Call Center Product"&sel=Workitems:Defect[AssetState='Closed'].Estimate.#Sum,Name,Workitems:Story[AssetState='Closed'].Estimate.#Sum
My Not working Code:
IAssetType storyType = services.getMeta().getAssetType("Timebox");
Query query = new Query(storyType, true);
IAttributeDefinition name = storyType.getAttributeDefinition("Name");
IAttributeDefinition defect_estimate = storyType.getAttributeDefinition("Workitems:Defect[AssetState='Closed'].Estimate.#Sum");
IAttributeDefinition story_estimate = storyType.getAttributeDefinition("Workitems:Story[AssetState='Closed'].Estimate.#Sum");
query.getSelection().add(name);
query.getSelection().add(defect_estimate);
query.getSelection().add(story_estimate);
//IFilterTerm activeSprint = new TokenTerm("State.Code='ACTV'");
IFilterTerm activeSprint = new TokenTerm("Schedule.ScheduledScopes.Name='Sample: Call Center Product'");
query.setFilter(activeSprint);
DefaultCategoryDataset dataset = new DefaultCategoryDataset( );
QueryResult result = services.retrieve(query);
Error i am getting:
Exception in thread "main" com.versionone.apiclient.exceptions.MetaException: Unknown AttributeDefinition: Timebox.Workitems:Defect[AssetState='Closed'].Estimate.#Sum
at com.versionone.apiclient.MetaModel.getAttributeDefinition(MetaModel.java:119)
at com.versionone.apiclient.AssetType.getAttributeDefinition(AssetType.java:96)
at v1_rest_intig.Example1.main(Example1.java:230)
what am i doing wrong??
any guidance is of great help
Thanks in advance

You are using the right attribute definition, but apparently, at some version of the API they stop translating symbols, like brakets '[' into the URL encoding ( '%5B' for open bracket) and therefore the resulting error message.
Please, try this instead:
Workitems:Defect%5BAssetState=%27128%27%5D
for your attributes definition for the Defect/Story AssetType.
Let me know if this works.
TIA,

find amazon categories using python

I would like to get the categories of the amazon ,I am planning to scrap not to use API.
I have scrapped the http://www.amazon.com.I have scraped all the categories and sub-categories under Shop By Department drop down .I have created a web service to do this The code is here
#route('/hello')
def hello():
text=list();
link=list();
req = urllib2.Request("http://www.amazon.com",
headers={"Content-Type": "application/json"})
html=urllib2.urlopen(req).read()
soup = BeautifulSoup(html)
last_page = soup.find('div', id="nav_subcats")
for elm in last_page.findAll('a'):
texts = elm.text
links = elm.get('href')
links = links.partition("&node=")[2]
text.append(texts)
link.append(links)
alltext=list();
for i,j in zip(text,link):
alltext.append({"name":i,"id":j})
response.content_type = 'application/json'
print(alltext)
return dumps(alltext)
run(host='localhost', port=8080, debug=True)
I am passing the category name and category id as a JSON object to one of my members to pass it to the API to get the product listing for each category
It is written in JAVA.Here is the code
for (int pageno = 1; pageno <= 10; pageno++) {
String page = String.valueOf(pageno);
String category_string = selectedOption.get("category_name").toString();
String category_id = selectedOption.get("category_id").toString();
final Map<String, String> params = new HashMap<String, String>(3);
params.put(AmazonClient.Op.PARAM_OPERATION, "ItemSearch");
params.put("SearchIndex", category_string);
params.put("BrowseNodeId", category_id);
params.put("Keywords", category_string);
params.put("ItemPage", page);
System.out.println(client.documentToString(client.getXml(params)));
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
Document doc = null;
DocumentBuilder db = dbf.newDocumentBuilder();
InputStream is = client.getInputStream(params);
doc = db.parse(is);
NodeList itemList = doc.getElementsByTagName("Items");
But i am getting this error when i pass the category id as the BrowseNodeId and category name as keyword and search index.
For example
Search Index and Keyword -Amazon Instant Video
BrowseNodeId-2858778011
The value you specified for SearchIndex is invalid. Valid values include [ 'All','Apparel',...................................reless','WirelessAccessories' ].
I would like to know from which amazon url i will get all the categories and its browse nodes
Thank you

I have never looked at Amazon's API before, so this is just a guess but, based on the error message it would seem that "Amazon Instant Video" is not a valid search index. Just because it is there in the drop-down list, doesn't necessarily mean that it is a valid search index.
Here's a list of search indices for US: http://docs.aws.amazon.com/AWSECommerceService/latest/DG/USSearchIndexParamForItemsearch.html . I don't know how up to date it is, but "Amazon Instant Video" does not appear on the list. The error message does include a list of valid search index values, and these do appear to correspond to the above list.
For other locales look here : http://docs.aws.amazon.com/AWSECommerceService/latest/DG/APPNDX_SearchIndexParamForItemsearch.html
I don't think that this is a coding problem per se.
You might like to take a look at python-amazon-product-api. The API might be useful to you, and the documentation might give you some ideas.

Adding a markup outside an existing tag regex

I have an xml alone, I wanted to add a markup outside of a tag called scl
so for example
<documents>
<rnl></rnl>
<snl></snl>
<scl>This is scl</scl>
</documents>
So <scl>This is scl</scl> will be transformed to <credit><scl>This is scl</scl></credit>
How do I modify the below to achieve this ? So far I have tried.
String content5 = "<scl>this is scl</scl>";
System.out.println(content5.replaceAll("^(?:<scl>)(?:.*?)(?:</scl>)", "<credit>$1</credit>" ));

one way is to do like:
String content5 = "<scl>this is scl</scl>";
System.out.println(content5.replace("<scl>", "<credit><scl>").replace("</scl>", "</scl></credit>"));

Here you go:
String content5 =
"<scl>this is scl</scl>\n" +
"<blah>this isn't</blah>\n" +
"<scl>this is scl</scl>\n" +
"<blah>this isn't</blah>\n";
System.out.println(content5.replaceAll("(?:<scl>)(?:.*?)(?:</scl>)", "<credit>$0</credit>" ));
output:
<credit><scl>this is scl</scl></credit>
<blah>this isn't</blah>
<credit><scl>this is scl</scl></credit>
<blah>this isn't</blah>

Try this
StringBuilder sb = new StringBuilder(content5);
sb.insert(sb.indexOf("<scl>"), "<credit>");
sb.insert(sb.indexOf("</scl>") + "</scl>".length(), "</credit>");
String result = sb.toString();

You can completely change your way of thinking about the problem and use proper library to handle XML Dom4j. Very easy to learn, very simple and straightforward to use. Everything you need is explained in this link.

I cannot access an object property

I'm using Java and Jena API.
I have a class Marriage which have 3 Object Properties called "hasHusband", "Haswife" and "dateOfMarriage". The first two are associated with a class Person which has the datatypeproperties like hasFirstName, hasLastName, dateOfBirth....
I'd like to access the objects properties "Haswife" and "hasHusband" and then the wife's first name and the husband's first name.
Here is how that is represented in my rdf file:
(...)
<j.0:FAMmariage rdf:about=http://www.fam.com/FAM#BrunoCatherine>
<j.0:FAMaDateMariage>25/07/2011</j.0:FAMaDateMariage>
<j.0:FAMhasWife>
<rdf:Description rdf:about="http://www.fam.com/FAM#Catherine20/03/1982">
<j.0:FAMDateOfBirth>20/03/1980</j.0:FAMDateOfBirth>
<j.0:FAMHasName>Gomez</j.0:FAMHasName>
<j.0:FAMHasFirstName>Catherine</j.0:FAMHasFirstName>
</rdf:Description>
</j.0:FAMHasWife>
<j.0:FAMHusband>
<rdf:Description rdf:about="http://www.fam.com/FAM# Bruno15/06/1980 ">
<j.0:FAMaDateOfBirth>15/06/1980 </j.0:FAMDateOfBirth>
<j.0:FAMHasName>Jeandet </j.0:FAMHasName>
<j.0:FAMHasFirstName>Bruno</j.0:FAMHasFirstName>
</rdf:Description>
</j.0:FAMHusband>
</j.0:FAMmariage>
(...)
I tried like this but it still does not works:
StmtIterator iter = onto.model.listStatements();
while (iter.hasNext()) {
Statement stmt = iter.nextStatement();
Resource subject = stmt.getSubject();
Property predicate = stmt.getPredicate();
RDFNode object = stmt.getObject();
if (predicate.equals(onto.hasWife))
{
System.out.print(" " + object.toString() + " ");
}
}
Can you please tell me what's wrong?
Thanks
EDITED
More useful details:
(...)
person = model.createClass(uriBase+"person");
person.addSubClass(man);
person.addSubClass(woman);
marriage = model.createClass(uriBase+"marriage");
(...)
hasHusband = model.createObjectProperty(uriBase+"hasHusband");
hasHusband.setDomain(marriage);
hasHusband.setRange(man);
hasWife = model.createObjectProperty(uriBase+"hasWife");
hasWife.setDomain(marriage);
hasWife.setRange(woman);
hasFirstName = model.createDatatypeProperty(uriBase+"hasFirstName");
hasFirstName.setDomain(person);
hasFirstName.setRange(XSD.xstring);
(...)

does that work ?
ont.hasWife=model.createProperty("your namespace URI here","FAMhasWife");
StmtIterator iter = onto.model.listStatements(null,onto.hasWife,(RDFNode)null);
while (iter.hasNext()) {
Statement stmt = iter.nextStatement();
System.out.print(stmt.getObject().toString());
}

With the information you provide it's impossible to know what's the problem exactly. It is possible that you made a mistake in one of your URIs, either in your data or in your programme. Your question is inconcistent wrt naming terms.
May I suggest a few things? First, use the datatypes xsd:date or even xsd:dateTime for dates of marriage and dates of birth. Use the YYYY-MM-DD pattern for naming entities as this sorts them chronologically if sorted lexicographically. Use consistent naming convention, e.g., hasWife and hasHusband instead of hasWife and husband. "Marriage" has two "r" in English. Use a meaningful prefix when presenting your data to people. Better, use a human friendly syntax such as Turtle.
fam:BrunoCatherine a fam:Marriage;
fam:dateOfMarriage "2011-07-25"^^xsd:date;
fam:hasWife fam:Catherine1982-03-20;
fam:hasHusband fam:Bruno1980-06-15 .
fam:Catherine1982-03-20 fam:dateOfBirth "1982-03-20"^^xsd:date;
fam:hasLastName "Gomez";
fam:hasFirstName "Catherine" .
fam:Bruno1980-06-15 fam:dateOfBirth "1980-06-15"^^xsd:date;
fam:hasLastName "Jeandet";
fam:hasFirstName "Bruno" .
Doesn't it look nicer?