We Keep Coding

OWASP library custom policy - adding spaces where removing html

I am currently trying to use the OWASP library to remove some html from a string. A string that I have is a list:
one
two
three
Which in markup, the string looks like "<ul><li>one</li><li>two</li><li>three</li></ul>".
When I use the OWSAP library with a policy like:
PolicyFactory policy = new HtmlPolicyBuilder()
.allowUrlProtocols("https")
.allowStandardUrlProtocols()
.requireRelNofollowOnLinks()
.toFactory();
OSWAP converts the list to: "onetwothree" . I instead would like to add spaces between the list items, and be able to convert the string to "one two three". I was wondering how / if there is a way to do this with OWSAP? I am new to using this so any advice is appreciated!

This is how it works, in case someone in future faced the same issue
It's in kotlin but pretty simple to be converted to java
private class SeparateTextPostProcessor(htmlStreamEventReceiver: HtmlStreamEventReceiver, val separator: String = " ") :
HtmlStreamEventReceiverWrapper(htmlStreamEventReceiver) {
override fun text(elem: String) {
if (elem.isNotEmpty()) {
// adds a space between elements
underlying.text("$elem$separator")
} else {
underlying.text(elem)
}
}
}
fun String.stripHTML(): String {
val policy =
HtmlPolicyBuilder().withPostprocessor { SeparateTextPostProcessor(it) }.toFactory()
val sanitized = policy.sanitize(this)
// Sanitized text will contain additional escaped characters.
return sanitized.trim()
}

Simple Code to Copy Same Name Properties?

I have an old question sustained in my mind for a long time. When I was writing code in Spring, there are lots dirty and useless code for DTO, domain objects. For language level, I am hopeless in Java and see some light in Kotlin. Here is my question:
Style 1 It is common for us to write following code (Java, C++, C#, ...)
// annot: AdminPresentation
val override = FieldMetadataOverride()
override.broadleafEnumeration = annot.broadleafEnumeration
override.hideEnumerationIfEmpty = annot.hideEnumerationIfEmpty
override.fieldComponentRenderer = annot.fieldComponentRenderer
Sytle 2 Previous code can be simplified by using T.apply() in Kotlin
override.apply {
broadleafEnumeration = annot.broadleafEnumeration
hideEnumerationIfEmpty = annot.hideEnumerationIfEmpty
fieldComponentRenderer = annot.fieldComponentRenderer
}
Sytle 3 Can such code be even simplified to something like this?
override.copySameNamePropertiesFrom (annot) { // provide property list here
broadleafEnumeration
hideEnumerationIfEmpty
fieldComponentRenderer
}
First Priority Requirments
Provide property name list only one time
The property name is provided as normal code, so as to we can get IDE auto complete feature.
Second Priority Requirements
It's prefer to avoid run-time cost for Style 3. (For example, 'reflection' may be a possible implementation, but it do introduce cost.)
It's prefer to generated code like style1/style2 directly.
Not care
The final syntax of Style 3.
I am a novice for Kotlin language. Is it possible to use Kotlin to define somthing like 'Style 3' ?

It should be pretty simple to write a 5 line helper to do this which even supports copying every matching property or just a selection of properties.
Although it's probably not useful if you're writing Kotlin code and heavily utilising data classes and val (immutable properties). Check it out:
fun <T : Any, R : Any> T.copyPropsFrom(fromObject: R, vararg props: KProperty<*>) {
// only consider mutable properties
val mutableProps = this::class.memberProperties.filterIsInstance<KMutableProperty<*>>()
// if source list is provided use that otherwise use all available properties
val sourceProps = if (props.isEmpty()) fromObject::class.memberProperties else props.toList()
// copy all matching
mutableProps.forEach { targetProp ->
sourceProps.find {
// make sure properties have same name and compatible types
it.name == targetProp.name && targetProp.returnType.isSupertypeOf(it.returnType)
}?.let { matchingProp ->
targetProp.setter.call(this, matchingProp.getter.call(fromObject))
}
}
}
This approach uses reflection, but it uses Kotlin reflection which is very lightweight. I haven't timed anything, but it should run almost at same speed as copying properties by hand.
Now given 2 classes:
data class DataOne(val propA: String, val propB: String)
data class DataTwo(var propA: String = "", var propB: String = "")
You can do the following:
var data2 = DataTwo()
var data1 = DataOne("a", "b")
println("Before")
println(data1)
println(data2)
// this copies all matching properties
data2.copyPropsFrom(data1)
println("After")
println(data1)
println(data2)
data2 = DataTwo()
data1 = DataOne("a", "b")
println("Before")
println(data1)
println(data2)
// this copies only matching properties from the provided list
// with complete refactoring and completion support
data2.copyPropsFrom(data1, DataOne::propA)
println("After")
println(data1)
println(data2)
Output will be:
Before
DataOne(propA=a, propB=b)
DataTwo(propA=, propB=)
After
DataOne(propA=a, propB=b)
DataTwo(propA=a, propB=b)
Before
DataOne(propA=a, propB=b)
DataTwo(propA=, propB=)
After
DataOne(propA=a, propB=b)
DataTwo(propA=a, propB=)

Property file based conditional patterns in java

I have a property file (a.txt) which has the values (Example values given below) like below
test1=10
test2=20
test33=34
test34=35
By reading this file, I need to produce an output like below
value = 35_20_34_10
which means => I have a pattern like test34_test2_test33_test1
Note, If the 'test33' has any value other than 34 then I need to produce the value like below
value = 35_20_10
which means => I have a pattern like test34_test2_test1
Now my problem is, every time when the customer is making the change in the logic, I am making the change in the code. So what I expect is, I want to keep the logic (pattern) in another property file so I will be sending the two inputs to the util (one input is the property file (A.txt) another input will be the 'pattern.txt'),
My util has to be compare the A.txt and the business logic 'pattern.txt' and produce the output like
value = 35_20_34_10 (or)
value = 35_20_10
If there an example for such pattern based logic as I expect?
Any predefined util / java class does this?
Any help would be Great.
thanks,
Harry

First of all, svasa's answer makes a lot of sense, but covers different level of
abstraction. I recommend you read his answer too, that pattern should
be useful.
You may wanna look at Apache Velocity and FreeMarker libraries to see how they structure their API.
Those are template engines - they usually have some abstraction of pattern or format, and abstraction of variable/value binding (or namespace, or source). You can render a template by binding it with a binding/namespace, which yields the result.
For example, you may wanna have a pattern "<a> + <b>", and binding that looks like a map: {a: "1", b: "2"}. By binding that binding to that pattern you'll get "1 + 2", when interpreting <...> as variables.
You basically load the pattern from your pattern.txt, then load your data file A.txt (for example, by treating it as properties and using Properties class) and construct binding based on these properties. You'll get your output and possibility to customize the pattern all the time.

You may call the sequences like test34_test2_test33_test1 as a pattern, let me call them as constraints when building something.
To me this problem best fits into a
builder pattern.
When building the value you want, you tell the builder that these are my constraints(pattern) and these are my original properties like below:
new MyPropertiesBuilder().setConstraints(constraints).setProperties(original).buildValue();
Details:
Set some constraints in a separate file where you specify the order of the properties and their values like :
test34=desiredvalue-could-be-empty
test2=desiredvalue-could-be-empty
test33=34
test1=desiredvalue-could-be-empty
The builder goes over the constraints in the order specified, but get the values from the original properties and build the desired string.
One way to achieve your requirement through builder pattern is to define classes like below :
Interface:
public interface IMyPropertiesBuilder
{
public void setConstraints( Properties properties );
public void setProperties( Properties properties );
public String buildValue();
}
Builder
public class MyPropertiesBuilder implements IMyPropertiesBuilder
{
private Properties constraints;
private Properties original;
#Override
public void setConstraints( Properties constraints )
{
this.constraints = constraints;
}
#Override
public String buildValue()
{
StringBuilder value = new StringBuilder();
Iterator it = constraints.keySet().iterator();
while ( it.hasNext() )
{
String key = (String) it.next();
if ( original.containsKey( key ) && constraints.getProperty( key ) != null && original.getProperty( key ).equals( constraints.getProperty( key ) ) )
{
value.append( original.getProperty( key ) );
value.append( "_" );
}
}
return value.toString();
}
#Override
public void setProperties( Properties properties )
{
this.original = properties;
}
}
User
public class MyPropertiesBuilderUser
{
private Properties original = new Properties().load(new FileInputStream("original.properties"));;
private Properties constraints = new Properties().load(new FileInputStream("constraints.properties"));
public String getValue()
{
String value = new MyPropertiesBuilder().setConstraints(constraints).setProperties(original).buildValue();
}
}

Java SnakeYaml - prevent dumping reference names

I have the following method which I use to get the object converted to yaml representation (which I can eg. print to console)
#Nonnull
private String outputObject(#Nonnull final ObjectToPrint packageSchedule) {
DumperOptions options = new DumperOptions();
options.setAllowReadOnlyProperties(true);
options.setPrettyFlow(true);
return new Yaml(new Constructor(), new JodaTimeRepresenter(), options).dump(ObjectToPrint);
}
All is good, but for some object contained within ObjectToPrint structure I get something like reference name and not the real object content, eg.
!!com.blah.blah.ObjectToPrint
businessYears:
- businessYearMonths: 12
ppiYear: &id001 {
endDate: 30-06-2013,
endYear: 2013,
startDate: 01-07-2012,
startYear: 2012
}
ppiPeriod:
ppiYear: *id001
endDate: 27-03-2014
startDate: 21-06-2013
units: 24.000
number: 1
As you can see from the example above I have ppiYear object printed (marked as $id001) and the same object is used in ppiPeriod but only the reference name is printed there, not the object content.
How to print the object content everytime I use that object within my structure, which I want to be converted to yaml (ObjectToPrint).
PS. It would be nice not to print the reference name at all (&id001) but thats not crucial

This is because you reference the same object at different places. To avoid this you need to create copies of those objects.
Yaml does not have a flag to switch this off, because you might get into endless loops in case of cyclic references.
However you might tweak Yaml source code to ignore double references:
look at Serializer line ~170 method serializeNode:
...
if ( this.serializedNodes.contains(node) ) {
this.emmitter.emit( new AliasEvent( ... ) );
} else {
serializedNodes.add(node); // <== Replace with myHook(serializedNodes,node);
...
void myHook(serializedNodes,node) {
if ( node's class != myClass(es) to avoid ) {
serializedNodes.add(node);
}
if you find a way to avoid Yaml to put nodes into the serializedNodes collection, your problem will be solved, however your programm will loop endless in case of cyclic references.
Best solution is to add a hook which avoids registering only the class you want to be written plain.

As a way to do this without changing the SnakeYAML source code, you can define:
public class NonAnchorRepresenter extends Representer {
public NonAnchorRepresenter() {
this.multiRepresenters.put(Map.class, new RepresentMap() {
public Node representData(Object data) {
return representWithoutRecordingDescendents(data, super::representData);
}
});
}
protected Node representWithoutRecordingDescendents(Object data, Function<Object,Node> worker) {
Map<Object,Node> representedObjectsOnEntry = new LinkedHashMap<Object,Node>(representedObjects);
try {
return worker.apply(data);
} finally {
representedObjects.clear();
representedObjects.putAll(representedObjectsOnEntry);
}
}
}
and use it as e.g. new Yaml(new SafeConstructor(), new NonAnchorRepresenter());
This only does maps, and it does use anchors when it has to, i.e. where a map refers to an ancestor. It would need similar extensions for sets and lists if required. (In my case it was empty maps that were the biggest offender.)
(It would be more easily handled in the SnakeYAML codebase on Representer.representData looking at an option e.g. setAllowAnchors defaulting to true, with the logic above to reset representedObjects after recursing if disallowed. I take the point at https://stackoverflow.com/a/18419489/109079 about the possibility of endless loops but it would be straightforward using this strategy to detect any reference to a parent map and fail fast.)

Another solution which i came by is using io.circle instead snake-yaml if it possible.
Scala code:
private[this] def removeAnchors(configYaml: String): String = {
val withoutAnchorsObj = io.circe.yaml.parser.parse(configYaml).valueOr(throw _)
val withoutAnchorString = io.circe.yaml.Printer(dropNullKeys = true, mappingStyle = Printer.FlowStyle.Block).pretty(withoutAnchorsObj)
logger.info(s"Removed Anchors from configYaml: $configYaml, result: $withoutAnchorString")
withoutAnchorString
}
build.sbt:
val circeVersion = "0.12.0"
libraryDependencies ++= Seq(
"io.circe" %% "circe-yaml" % circeVersion,
"io.circe" %% "circe-parser" % circeVersion,
)

Writing and Reading Repeated Extended Fields in Protobuf

I'm trying to use extensions with Google's Protocol Buffers. I've got one "main" proto file and several other proto files that "extend" the main one.
In my java code, I'm not 100% sure how to add to a repeating message correctly. When I run the java code I've written, the toString() method shows that the extension attributes added, but it doesn't decode properly (probably because I ran a build() call on the added Collar object).
How should I properly add repeating elements to extended items in a proto file?
File1.proto
package protocol_buffer;
option java_outer_classname = "PetClass";
message Pet {
optional string pet_name = 1;
extensions 100 to 199;
}
File2.proto
import "File1.proto";
option java_outer_classname = "CollarClass";
message Collars {
optional string collar_type = 1;
optional string collar_color = 2;
}
extend pet {
repeated Collars collar = 100;
}
MyFile.java
Pet pet = Pet.newBuilder()
.setPetName("Fido")
.addExtension(CollarClass.collar,
Collar.newBuilder()
.setCollarType("round")
.setCollarColor("brown")
.build()
)
.build();
System.out.println(pet.toString());

I figured out my problem. I was correctly adding the extension to "Pet". When parsing a protobuf byte array, you need to add an extension registry so that the function knows what extensions to parse.
ExtensionRegistry registry = ExtensionRegistry.newInstance();
registry.add(CollarClass.collar);
Pet pet = Pet.parseFrom(new FileInputStream(<some file>),registry);