We Keep Coding

Get lines of code in src folder skipping comments and empty lines

Hi I already have something to get the lines of code but still it out puts the count having empty lines and comments counted.
git ls-files | grep "\.java$" | xargs wc -l
Can you modify this to skip comments and blank lines..?
Thanks in advance

Try CLOC, it can lists numbers in great detail.
You need to install CLOC first using syntax brew install cloc
cloc $(git ls-files)
Sample output for reference:
20 text files.
20 unique files.
6 files ignored.
http://cloc.sourceforge.net v 1.62 T=0.22 s (62.5 files/s, 2771.2 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Javascript 2 13 111 309
JSON 3 0 0 58
HTML 2 7 12 50
Handlebars 2 0 0 37
CoffeeScript 4 1 4 12
SASS 1 1 1 5
-------------------------------------------------------------------------------
SUM: 14 22 128 471
-------------------------------------------------------------------------------

Most test coverage tools also calculate LOC. For instance, Jacoco spits this out to Jenkins.

Regexp for german phone number format

I try to get phone numbers from string in german format. But I don't get it to full run. The input text is a full HTML-Page with lots of content, not only the numbers.
Possible Formats:
(06442) 3933023
(02852) 5996-0
(042) 1818 87 9919
06442 / 3893023
06442 / 38 93 02 3
06442/3839023
042/ 88 17 890 0
+49 221 549144 – 79
+49 221 - 542194 79
+49 (221) - 542944 79
0 52 22 - 9 50 93 10
+49(0)121-79536 - 77
+49(0)2221-39938-113
+49 (0) 1739 906-44
+49 (173) 1799 806-44
0173173990644
0214154914479
02141 54 91 44 79
01517953677
+491517953677
015777953677
02162 - 54 91 44 79
(02162) 54 91 44 79
I have tried:
$regex = '~(?:\+?49|0)(?:\s*\d{3}){2}\s*\d{4,10}~';
if(preg_match_all($regex, $input_imprint , $matches)){
print_r($matches);
}
But it doesn't match only a few formats. I have no idea to do it.

Here is a regex to match all your formats.
I would suggest then to replace all unwanted characters and you got your desired result.
(\(?([\d \-\)\–\+\/\(]+)\)?([ .\-–\/]?)([\d]+))
If you need a minimum length to match your numbers, use this:
(\(?([\d \-\)\–\+\/\(]+){6,}\)?([ .\-–\/]?)([\d]+))
https://regex101.com/r/CAVex8/143
updated, thanks for the suggestion #Willi Mentzel

[0-9]*\/*(\+49)*[ ]*(\([0-9]+\))*([ ]*(-|–)*[ ]*[0-9]+)*
Check this link: https://regex101.com/r/CAVex8/1
May introduce some false positives.

This one solved my problem (extracting phone numers from emails):
r"\+?[0-9]+([0-9]|\/|\(|\)|\-| ){10,}"
A plus sign optional at the front, followed by at least 1 number, followed by at least 10 numbers or delimiting characters such as /, (, ) or - or a space.
(There is no official "smallest number of digits" for a telephone number, but I assume they are all at least 11 digits long)
I'm adding this because #Kakul 's solution matched any lien of my text, and using #despecial 's my code would not terminate. (I am guessing it is too computationally expensive for my pc)

This is no solution for the asked question, just an advice for matching phonenumbers!
If you are about to store telephone numbers for you first time, then limit the amount of different accepted formats. Get rid of these for example:
(06442) 3933023
042/ 88 17 890 0
+49(0)121-79536 - 77
02162 - 54 91 44 79
Why?
You need to test more possible ways of inputting an invalid value.
Those formats you absolutely need to concider according to DIN 5008:
0873 376461
03748 37682358
05444 347687-350
0764 812632-41
0180 2 12334
0800 5 23234213
+49 30 3432622-113
0179 1111111
Here is what I came up with: Regex
^(([+]{1}[1-9]{1}[0-9]{0,2}[ ]{1}([1-9]{1}[0-9]{1,4}){1}[ ]{1}([1-9]{1}[0-9]{2,6}){1}([ -][0-9]{1,5})?)|([0]{1}[1-9]{1}[0-9]{1,4}[ ]{1}[0-9]{1,8}([ -][0-9]{1,8})?)?)
Positives:
06429 1111
06901 306180
06429 231
0800 3301000
0179 1111111
0873 376461
03748 37682358
05444 347687-350
0764 812632-41
0180 2 12334
0800 5 23234213
+49 6429 1111
+49 39857 2530
+55 11 2666-0054
+300 11 2666-0054
+49 641 20106 0
+49 641 20106
+49 30 3432622-113
Negatives:
++49 157 184977
+300 11 0000-0000
(06442) 3933023
(02852) 5996-0
(042) 1818 87 9919
06442 / 3893023
06442 / 38 93 02 3
06442/3839023
042/ 88 17 890 0
+49 221 - 542194 79
+49 (221) - 542944 79
0 52 22 - 9 50 93 10
+49(0)121-79536 - 77
+49(0)2221-39938-113
+49 (0) 1739 906-44
+49 (173) 1799 806-44
0173173990644
0214154914479
01517953677
+491517953677
015777953677
02162 - 54 91 44 79
(02162) 54 91 44 79
saddsadasdasd
asdasd
asdasd asdasd asd
asdasd
kjn asohas asdoiasd
23434 234 234 23
323
23434 234----234
///// ----
// id8834 3493934 //

Hey i have a little enhancement for despecial‘s Regex:
(\(?([\d \-\)\–\+\(]+\/?){6,}\)?([ .\-–\/]?)([\d]+))
It filters numbers that have too high occurrencies of /

I got a different result when I retrained the sentiment model with Stanford CoreNLP to compare with the related paper's result

I downloaded stanford-corenlp-full-2015-12-09.
And I created a training model with the following command:
java -mx8g edu.stanford.nlp.sentiment.SentimentTraining -numHid 25 -trainPath train.txt -devPath dev.txt -train -model model.ser.gz
When I finished training, I found many files in my directory.
the model list
Then I used the evaluation tool from the package and I ran the code like this:
java -cp * edu.stanford.nlp.sentiment.Evaluate -model model-0024-79.82.ser.gz -treebank test.txt
The test.txt was from trainDevTestTrees_PTB.zip. This is the result about code:
F:\trainDevTestTrees_PTB\trees>java -cp * edu.stanford.nlp.sentiment.Evaluate -model model-0024-79.82.ser.gz -treebank test.txt
EVALUATION SUMMARY
Tested 82600 labels
65331 correct
17269 incorrect
0.790932 accuracy
Tested 2210 roots
890 correct
1320 incorrect
0.402715 accuracy
Label confusion matrix
Guess/Gold 0 1 2 3 4 Marg. (Guess)
0 551 340 87 32 6 1016
1 956 5348 2476 686 191 9657
2 354 2812 51386 3097 467 58116
3 146 744 2525 6804 1885 12104
4 1 11 74 379 1242 1707
Marg. (Gold) 2008 9255 56548 10998 3791
0 prec=0.54232, recall=0.2744, spec=0.99423, f1=0.36442
1 prec=0.5538, recall=0.57785, spec=0.94125, f1=0.56557
2 prec=0.8842, recall=0.90871, spec=0.74167, f1=0.89629
3 prec=0.56213, recall=0.61866, spec=0.92598, f1=0.58904
4 prec=0.72759, recall=0.32762, spec=0.9941, f1=0.4518
Root label confusion matrix
Guess/Gold 0 1 2 3 4 Marg. (Guess)
0 50 60 12 9 3 134
1 161 370 147 94 36 808
2 31 103 102 60 32 328
3 36 97 123 305 265 826
4 1 3 5 42 63 114
Marg. (Gold) 279 633 389 510 399
0 prec=0.37313, recall=0.17921, spec=0.9565, f1=0.24213
1 prec=0.45792, recall=0.58452, spec=0.72226, f1=0.51353
2 prec=0.31098, recall=0.26221, spec=0.87589, f1=0.28452
3 prec=0.36925, recall=0.59804, spec=0.69353, f1=0.45659
4 prec=0.55263, recall=0.15789, spec=0.97184, f1=0.24561
Approximate Negative label accuracy: 0.638817
Approximate Positive label accuracy: 0.697140
Combined approximate label accuracy: 0.671925
Approximate Negative root label accuracy: 0.702851
Approximate Positive root label accuracy: 0.742574
Combined approximate root label accuracy: 0.722680
The accuracy about fine-grained and positive/negative was quite different from the paper "Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y. and Potts, C., 2013, October. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP) (Vol. 1631, p. 1642)."
The paper states the accuracy about fine-grained and positive/negative is higher than mine.
The records in the paper
Were there any problems with my operation? Why was my result different from the paper?

The short answer is that the paper used a different system written in Matlab. The Java system does not match the paper. Though we do distribute the binary model we trained in Matlab with the English models jar. So you can RUN the binary model with Stanford CoreNLP, but you cannot TRAIN a binary model with similar performance with Stanford CoreNLP at this time.

Count Unique Text and value count

I would like to get the result unique count value/text/ etc
A B
2 BADER 111
3 FAISA 112
4 NASSE 113
5 NASSE 113
6 MOHS 121
7 ASI 122
8 AHME 100
9 AHME 100
10 AHME 100
11 ASI 122
RESULT AS BELOW.
A B
2 BADER 111
3 FAISA 112
4 NASSE 113
5 NASSE 113
6 MOHS 121
7 ASI 122
8 AHME 100
9 AHME 100
10 AHME 100
11 ASI 122
6 6

For the number of different values in A2:A11 try this formula
=SUMPRODUCT((A2:A11<>"")/COUNTIF(A2:A11,A2:A11&""))
That will work for numeric or text values

How to figure out that 95 pecentile of time calls came back within certain milliseconds

I am working on a project in which I am supposed to do profiling on our REST Service. And after that figure it out how much is the 90 percentile or 95 percentile of our service. Meaning how much time calls is taking in 90% of time or 95% percent of time
Below is the histogram I have created by profiling my service-
0 came back between 1 and 2 ms
0 came back between 3 and 4 ms
0 came back between 5 and 8 ms
0 came back between 9 and 16 ms
0 came back between 17 and 32 ms
2205 came back between 33 and 64 ms
141 came back between 65 and 128 ms
50 came back greater than 128 ms
From the above histogram it means, 2205 calls came back between 33 and 64 ms, 141 calls came back between 65 and 128 ms.
So now I am trying to figure out how to calculate what is the 90 percentile or 95 percentile of this? Means 90 percentage of time calls came back in how many milliseconds?
I have a map as well for the above histogram, if we cannot figure out this percentile from the above histogram, then we can use this Map as well.
From this Map only, I am creating above histogram-
{213=1, 114=2, 185=1, 131=1, 40=145, 67=8, 49=35, 537=2, 164=1, 565=1,
55=13, 96=1, 546=1, 117=1, 68=10, 62=6, 83=1, 34=333, 41=108, 179=1,
48=48, 111=1, 129=1, 69=11, 33=1, 173=1, 61=8, 541=1, 74=7, 180=2,
42=78, 47=46, 56=11, 84=2, 70=12, 228=1, 273=1, 46=52, 102=1, 225=1,
81=2, 181=1, 563=1, 549=1, 137=1, 73=3, 235=1, 53=17, 90=1, 36=190,
118=1, 45=78, 35=267, 72=9, 63=16, 54=11, 271=1, 189=1, 209=1, 175=4,
51=23, 203=2, 37=186, 58=5, 196=2, 237=1, 86=3, 44=81, 64=15, 92=3,
224=1, 71=8, 251=1, 52=12, 78=3, 43=75, 147=1, 133=1, 580=1, 57=11,
263=1, 566=1, 85=1, 243=1, 38=161, 559=1, 80=3, 132=1, 194=1, 107=6,
65=5, 183=1, 222=1, 93=1, 60=12, 231=1, 94=1, 66=12, 122=1, 39=135,
50=35, 76=1, 59=6, 104=1, 158=1, 113=1, 204=1, 87=1, 115=2}
In the above map, key is the number of milliseconds and value is total number of calls
So for example -
213=1
It means, 1 call came back in 213 milliseconds.
114=2
2 calls came back in 114 milliseconds.
Can anyone help me with this? Either I can create the percentile stuff from the histogram or the above Map as well.

If you are doing performance tuning then i recommend you Metrics, which tells you whats really going on.