I have a problem and I can't find what it's wrong with my code.
From beginning. I have application to recognize digits and math operators, the basic one +, -, /, *. I have a model ready which creates a graph. This graph is imported by an Android application. In my opinion, it should work but I don't know why it isn't?
Model.py
import tensorflow as tf
import numpy as np
import _pickle as pickle
from matplotlib import pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
# mnist_data = input_data.read_data_sets('MNIST_data', one_hot=True)
# read CROHME_test_data
with open("test.pickle", 'rb') as f:
data = pickle.load(f)
# np.random.shuffle(data)
CROHME_test_data = {"features": np.array([d["features"] for d in data]), "labels": np.array([d["label"] for d in data])}
# read CROHME_train_data
with open("train.pickle", 'rb') as f:
data = pickle.load(f)
# np.random.shuffle(data)
CROHME_train_data = {"features": np.array([d["features"] for d in data]),
"labels": np.array([d["label"] for d in data])}
# function for pulling batches from custom data
def next_batch(num, data):
idx = np.arange(0, len(data["features"]))
np.random.shuffle(idx)
idx = idx[:num]
features_shuffle = [data["features"][i] for i in idx]
labels_shuffle = [data["labels"][i] for i in idx]
return np.asarray(features_shuffle), np.asarray(labels_shuffle)
# Function to create a weight neuron using a random number. Training will assign a real weight later
def weight_variable(shape, name):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial, name=name)
# Function to create a bias neuron. Bias of 0.1 will help to prevent any 1 neuron from being chosen too often
def biases_variable(shape, name):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial, name=name)
# Function to create a convolutional neuron. Convolutes input from 4d to 2d. This helps streamline inputs
def conv_2d(x, W, name):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME', name=name)
# Function to create a neuron to represent the max input. Helps to make the best prediction for what comes next
def max_pool(x, name):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', name=name)
labels_number = 14
# A way to input images (as 784 element arrays of pixel values 0 - 1)
x_input = tf.placeholder(dtype=tf.float32, shape=[None, 784], name='x_input')
# A way to input labels to show model what the correct answer is during training
y_input = tf.placeholder(dtype=tf.float32, shape=[None, labels_number], name='y_input')
# First convolutional layer - reshape/resize images
# A weight variable that examines batches of 5x5 pixels, returns 32 features (1 feature per bit value in 32 bit float)
W_conv1 = weight_variable([5, 5, 1, 32], 'W_conv1')
# Bias variable to add to each of the 32 features
b_conv1 = biases_variable([32], 'b_conv1')
# Reshape each input image into a 28 x 28 x 1 pixel matrix
x_image = tf.reshape(x_input, [-1, 28, 28, 1], name='x_image')
# Flattens filter (W_conv1) to [5 * 5 * 1, 32], multiplies by [None, 28, 28, 1] to associate each 5x5 batch with the
# 32 features, and adds biases
h_conv1 = tf.nn.relu(conv_2d(x_image, W_conv1, name='conv1') + b_conv1, name='h_conv1')
# Takes windows of size 2x2 and computes a reduction on the output of h_conv1 (computes max, used for better prediction)
# Images are reduced to size 14 x 14 for analysis
h_pool1 = max_pool(h_conv1, name='h_pool1')
# Second convolutional layer, reshape/resize images
# Does mostly the same as above but converts each 32 unit output tensor from layer 1 to a 64 feature tensor
W_conv2 = weight_variable([5, 5, 32, 64], 'W_conv2')
b_conv2 = biases_variable([64], 'b_conv2')
h_conv2 = tf.nn.relu(conv_2d(h_pool1, W_conv2, name='conv2') + b_conv2, name='h_conv2')
# Images at this point are reduced to size 7 x 7 for analysis
h_pool2 = max_pool(h_conv2, name='h_pool2')
# First dense layer, performing calculation based on previous layer output
# Each image is 7 x 7 at the end of the previous section and outputs 64 features, we want 32 x 32 neurons = 1024
W_dense1 = weight_variable([7 * 7 * 64, 1024], name='W_dense1')
# bias variable added to each output feature
b_dense1 = biases_variable([1024], name='b_dense1')
# Flatten each of the images into size [None, 7 x 7 x 64]
h_pool_flat = tf.reshape(h_pool2, [-1, 7 * 7 * 64], name='h_pool_flat')
# Multiply weights by the outputs of the flatten neuron and add biases
h_dense1 = tf.nn.relu(tf.matmul(h_pool_flat, W_dense1, name='matmul_dense1') + b_dense1, name='h_dense1')
# Dropout layer prevents overfitting or recognizing patterns where none exist
# Depending on what value we enter into keep_prob, it will apply or not apply dropout layer
keep_prob = tf.placeholder(dtype=tf.float32, name='keep_prob')
# Dropout layer will be applied during training but not testing or predicting
h_drop1 = tf.nn.dropout(h_dense1, keep_prob, name='h_drop1')
# Readout layer used to format output
# Weight variable takes inputs from each of the 1024 neurons from before and outputs an array of 14 elements
W_readout1 = weight_variable([1024, labels_number], name='W_readout1')
# Apply bias to each of the 14 outputs
b_readout1 = biases_variable([labels_number], name='b_readout1')
# Perform final calculation by multiplying each of the neurons from dropout layer by weights and adding biases
y_readout1 = tf.add(tf.matmul(h_drop1, W_readout1, name='matmul_readout1'), b_readout1, name='y_readout1')
# Softmax cross entropy loss function compares expected answers (labels) vs actual answers (logits)
cross_entropy_loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y_input, logits=y_readout1))
# Adam optimizer aims to minimize loss
train_step = tf.train.AdamOptimizer(0.0001).minimize(cross_entropy_loss)
# Compare actual vs expected outputs to see if highest number is at the same index, true if they match and false if not
correct_prediction = tf.equal(tf.argmax(y_input, 1), tf.argmax(y_readout1, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
# Used to save the graph and weights
saver = tf.train.Saver()
# Run in with statement so session only exists within it
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
# Save the graph shape and node names to pbtxt file
tf.train.write_graph(sess.graph_def, '.', 'advanced_mnist.pbtxt', False)
# Train the model, running through data 20000 times in batches of 50
# Print out step # and accuracy every 100 steps and final accuracy at the end of training
# Train by running train_step and apply dropout by setting keep_prob to 0.5
for i in range(1000):
batch = next_batch(50, CROHME_train_data)
if i % 100 == 0:
train_accuracy = accuracy.eval(feed_dict={x_input: batch[0], y_input: batch[1], keep_prob: 1.0})
print("step %d, training accuracy %g" % (i, train_accuracy))
train_step.run(feed_dict={x_input: batch[0], y_input: batch[1], keep_prob: 0.5})
print("test accuracy %g" % accuracy.eval(feed_dict={x_input: CROHME_test_data["features"],
y_input: CROHME_test_data["labels"], keep_prob: 1.0}))
# Save the session with graph shape and node weights
saver.save(sess, './advanced_mnist.ckpt')
# Make a prediction of random image
img_num = np.random.randint(0, len(CROHME_test_data["features"]))
print("expected :", CROHME_test_data["labels"][img_num])
print("predicted :",
sess.run(y_readout1, feed_dict={x_input: [CROHME_test_data["features"][img_num]], keep_prob: 1.0}))
for j in range(28):
print(CROHME_test_data["features"][img_num][j * 28:(j + 1) * 28])
# this code can be used for image displaying
first_image = CROHME_test_data["features"][img_num]
first_image = np.array(first_image, dtype='float')
pixels = first_image.reshape((28, 28))
plt.imshow(pixels, cmap='gray')
plt.show()
Graph.py
import tensorflow as tf
from tensorflow.python.tools import freeze_graph, optimize_for_inference_lib
# Saving the graph as a pb file taking data from pbtxt and ckpt files and providing a few operations
freeze_graph.freeze_graph('advanced_mnist.pbtxt',
'',
True,
'advanced_mnist.ckpt',
'y_readout1',
'save/restore_all',
'save/Const:0',
'frozen_advanced_mnist.pb',
True,
'')
# Read the data form the frozen graph pb file
input_graph_def = tf.GraphDef()
with tf.gfile.Open('frozen_advanced_mnist.pb', 'rb') as f:
data = f.read()
input_graph_def.ParseFromString(data)
# Optimize the graph with input and output nodes
output_graph_def = optimize_for_inference_lib.optimize_for_inference(
input_graph_def,
['x_input', 'keep_prob'],
['y_readout1'],
tf.float32.as_datatype_enum)
# Save the optimized graph to the optimized pb file
f = tf.gfile.FastGFile('optimized_advanced_mnist.pb', 'w')
f.write(output_graph_def.SerializeToString())
MainActivity
package com.example.owl.advanced_mnist_android;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.view.View;
import android.widget.ImageView;
import android.widget.TextView;
import org.tensorflow.contrib.android.TensorFlowInferenceInterface;
public class MainActivity extends AppCompatActivity {
ImageView imageView;
TextView resultsTextView;
static {
System.loadLibrary("tensorflow_inference");
}
private static final String MODEL_FILE = "file:///android_asset/optimized_advanced_mnist.pb";
private static final String INPUT_NODE = "x_input";
private static final int[] INPUT_SIZE = {1, 784};
private static final String KEEP_PROB = "keep_prob";
private static final int[] KEEP_PROB_SIZE = {1};
private static final String OUTPUT_NODE = "y_readout1";
private TensorFlowInferenceInterface inferenceInterface;
private int imageIndex = 10;
private int[] imageResourceIDs = {
R.drawable.digit0,
R.drawable.digit1,
R.drawable.digit2,
R.drawable.digit3,
R.drawable.digit4,
R.drawable.digit5,
R.drawable.digit6,
R.drawable.digit7,
R.drawable.digit8,
R.drawable.digit9,
R.drawable.rsz_przechwytywanie,
};
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
imageView = (ImageView) findViewById(R.id.image_view);
resultsTextView = (TextView) findViewById(R.id.results_text_view);
inferenceInterface = new TensorFlowInferenceInterface();
inferenceInterface.initializeTensorFlow(getAssets(), MODEL_FILE);
}
public void loadImageAction(View view) {
imageIndex = (imageIndex == 10) ? 0 : imageIndex + 1 ;
imageView.setImageResource(imageResourceIDs[imageIndex]);
}
public void predictDigitAction(View view) {
float[] pixelBuffer = convertImage();
float[] results = predictDigit(pixelBuffer);
formatResults(results);
}
private void formatResults(float[] results) {
float max = 0;
float secondMax = 0;
int maxIndex = 0;
int secondMaxIndex = 0;
for (int i = 0; i < 15; i++) {
if (results[i] > max) {
secondMax = max;
secondMaxIndex = maxIndex;
max = results[i];
maxIndex = i;
} else if (results[i] < max && results[i] > secondMax) {
secondMax = results[i];
secondMaxIndex = i;
}
}
String output = "Model predicts: " + String.valueOf(maxIndex) +
", second choice: " + String.valueOf(secondMaxIndex);
resultsTextView.setText(output);
}
private float[] predictDigit(float[] pixelBuffer) {
inferenceInterface.fillNodeFloat(INPUT_NODE, INPUT_SIZE, pixelBuffer);
inferenceInterface.fillNodeFloat(KEEP_PROB, KEEP_PROB_SIZE, new float[] {0.5f});
inferenceInterface.runInference(new String[] {OUTPUT_NODE});
float[] outputs = new float[14];
inferenceInterface.readNodeFloat(OUTPUT_NODE, outputs);
return outputs;
}
private float[] convertImage() {
Bitmap bitmap = BitmapFactory.decodeResource(getResources(), imageResourceIDs[imageIndex]);
Bitmap scaledBitmap = Bitmap.createScaledBitmap(bitmap, 28, 28, true);
imageView.setImageBitmap(scaledBitmap);
int[] intArray = new int[784];
float[] floatArray = new float[784];
scaledBitmap.getPixels(intArray, 0, 28, 0, 0, 28, 28);
for (int i = 0; i < 784; i++) {
floatArray[i] = intArray[i] / -16777216;
}
return floatArray;
}
}
Everything looks good, but I have an error:
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.owl.advanced_mnist_android, PID: 6855
java.lang.IllegalStateException: Could not execute method for android:onClick
at android.support.v7.app.AppCompatViewInflater$DeclaredOnClickListener.onClick(AppCompatViewInflater.java:293)
at android.view.View.performClick(View.java:6891)
at android.widget.TextView.performClick(TextView.java:12651)
at android.view.View$PerformClick.run(View.java:26083)
at android.os.Handler.handleCallback(Handler.java:789)
at android.os.Handler.dispatchMessage(Handler.java:98)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6938)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)
Caused by: java.lang.reflect.InvocationTargetException
at java.lang.reflect.Method.invoke(Native Method)
at android.support.v7.app.AppCompatViewInflater$DeclaredOnClickListener.onClick(AppCompatViewInflater.java:288)
at android.view.View.performClick(View.java:6891)
at android.widget.TextView.performClick(TextView.java:12651)
at android.view.View$PerformClick.run(View.java:26083)
at android.os.Handler.handleCallback(Handler.java:789)
at android.os.Handler.dispatchMessage(Handler.java:98)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6938)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)
Caused by: java.lang.IllegalArgumentException: No Operation named [x_input] in the Graph
at org.tensorflow.Session$Runner.operationByName(Session.java:297)
at org.tensorflow.Session$Runner.feed(Session.java:115)
at org.tensorflow.contrib.android.TensorFlowInferenceInterface.addFeed(TensorFlowInferenceInterface.java:437)
at org.tensorflow.contrib.android.TensorFlowInferenceInterface.fillNodeFloat(TensorFlowInferenceInterface.java:186)
at com.example.owl.advanced_mnist_android.MainActivity.predictDigit(MainActivity.java:89)
at com.example.owl.advanced_mnist_android.MainActivity.predictDigitAction(MainActivity.java:63)
at java.lang.reflect.Method.invoke(Native Method)
at android.support.v7.app.AppCompatViewInflater$DeclaredOnClickListener.onClick(AppCompatViewInflater.java:288)
at android.view.View.performClick(View.java:6891)
at android.widget.TextView.performClick(TextView.java:12651)
at android.view.View$PerformClick.run(View.java:26083)
at android.os.Handler.handleCallback(Handler.java:789)
at android.os.Handler.dispatchMessage(Handler.java:98)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6938)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)
I am doing an exercise, which requires to create 20 long arrays with 10^18 numbers in 10^5 cells. Every time I am always using the same array simply overwriting it in this way:
array = new long [b];
The program stops working at around 19th iteration, because of overused provided resources by the 'Hackerrank' website. What are the ways to make this code more efficient? I was thinking that maybe I am not deleting previously used array and it overuses system parameters. Is there a better way to do this? This is done in www.hackerrank.com and it only shows that "Runtime time" error occurs, but doesn't show which one exactly. It runs fine without printing the answer, so it probably requires to use slightly less resources and I think most resources are required for array.
My code:
import java.io.*;
import java.util.*;
import java.util.stream.*;
public class Solution {
public static void main(String[] args) {
/* Enter your code here. Read input from STDIN. Print output to STDOUT. Your class should be named Solution. */
Scanner sc = new Scanner(System.in);
int t = sc.nextInt();
int b;
long n;
long k;
long []array;
int count;
long sum = 0;
for(int i = 0; i < t; i++){
n = sc.nextLong();
k = sc.nextLong();
b = sc.nextInt();
count = 0;
sum = 0;
array = new long [b];
for (int j = 0; j < b; j++){
array[b - 1 - j]=j+1;
sum +=(j+1);
}
while (true){
while (sum < n && count < b && k > b){
if ((sum + (k - count - array[count]) )< n) {
sum += k - count - array[count];
array[count]+=k - count - array[count];
}
else {
System.out.println (n - sum);
array[count]+= n - sum;
sum += n - sum;
}
if (array[count] == k - count) count++;
}
if (sum == n) {
for (int x = 0; x < b; x++){
System.out.print (array[x]);
if (x < b - 1) System.out.print(" ");
else System.out.print("\n");
}
break;
}
else {
System.out.println ("-1");
break;
}
}
}
}
}
Input:
20
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
999999995000050000 10000000000000 100000
All cases work separately.
The output is:
10000000000000 9999999999999 9999999999998 9999999999997 9999999999996 9999999999995 9999999999994 9999999999993 9999999999992 9999999999991 9999999999990 9999999999989 9999999999988 9999999999987 9999999999986 9999999999985 9999999999984 9999999999983 9999999999982 9999999999981 9999999999980 9999999999979 9999999999978 9999999999977 9999999999976 9999999999975 9999999999974 9999999999973 9999999999972 9999999999971 9999999999970 9999999999969 9999999999968 9999999999967 9999999999966 9999999999965 9999999999964 9999999999963 9999999999962 9999999999961 9999999999960 9999999999959 9999999999958 9999999999957 9999999999956 9999999999955 9999999999954 9999999999953 9999999999952 9999999999951 9999999999950 9999999999949 9999999999948 9999999999947 9999999999946 9999999999945 9999999999944 9999999999943 9999999999942 9999999999941 9999999999940 9999999999939 9999999999938 9999999999937 9999999999936 9999999999935 9999999999934 9999999999933 9999999999932 9999999999931 9999999999930 9999999999929 9999999999928 9999999999927 9999999999926 9999999999925 9999999999924 9999999999923 9999999999922 9999999999921 9999999999920 9999999999919 9999999999918 9999999999917 9999999999916 9999999999915 9999999999914 9999999999913 9999999999912 9999999999911 9999999999910 9999999999909 9999999999908 9999999999907 9999999999906 9999999999905 9999999999904 9999999999903 9999999999902 9999999999901 9999999999900 9999999999899 9999999999898 9999999999897 9999999999896 9999999999895 9999999999894 9999999999893 9999999999892 9999999999891 9999999999890 9999999999889 9999999999888 9999999999887 9999999999886 9999999999885 9999999999884 9999999999883 9999999999882 9999999999881 9999999999880 9999999999879 9999999999878 9999999999877 9999999999876 9999999999875 9999999999874 9999999999873 9999999999872 9999999999871 9999999999870 9999999999869 9999999999868 9999999999867 9999999999866 9999999999865 9999999999864 9999999999863 9999999999862 9999999999861 9999999999860 9999999999859 9999999999858 9999999999857 9999999999856 9999999999855 9999999999854 9999999999853 9999999999852 9999999999851 9999999999850 9999999999849 9999999999848 9999999999847 9999999999846 9999999999845 9999999999844 9999999999843 9999999999842 9999999999841 9999999999840 9999999999839 9999999999838 9999999999837 9999999999836 9999999999835 9999999999834 9999999999833 9999999999832 9999999999831 9999999999830 9999999999829 9999999999828 9999999999827 9999999999826 9999999999825 9999999999824 9999999999823 9999999999822 9999999999821 9999999999820 9999999999819 9999999999818 9999999999817 9999999999816 9999999999815 9999999999814 9999999999813 9999999999812 9999999999811 9999999999810 9999999999809 9999999999808 9999999999807 9999999999806 9999999999805 9999999999804 9999999999803 9999999999802 9999999999801 9999999999800 9999999999799 9999999999798 9999999999797 9999999999796 9999999999190 9999999999189 9999999999188 9999999999187 9999999999186 9999999999185 9999999999184 9999999999183 9999999999182 9999999999181 9999999999180 9999999999179 9999999999178 9999999999177 9999999999176 9999999999175 9999999999174 9999999999173 9999999999172 9999999999171 9999999999170 9999999999169 9999999999168 9999999999167 9999999999166 9999999999165 9999999999164 9999999999163 9999999999162 9999999999161 9999999999160 9999999999159 9999999999158 9999999999157 9999999999156 9999999999155 9999999999154 9999999999153 9999999999152 9999999999151 9999999999150 9999999999149 9999999999148 9999999999147 9999999999146 9999999999145 9999999999144 9999999999143 9999999999142 9999999999141 9999999999140 9999999999139 9999999999138 9999999999137 9999999999136 9999999999135 9999999999134 9999999999133 9999999999132 9999999999131 9999999999130 9999999999129 9999999999128 9999999999127 9999999999126 9999999999125 9999999999124 9999999999123 9999999999122 9999999999121 9999999999120 9999999999119 9999999999118 9999999999117 9999999999116 9999999999115 9999999999114 9999999999113 9999999999112 9999999999111 9999999999110 9999999999109 9999999999108 9999999999107 9999999999106 9999999999105 9999999999104 9999999999103 9999999999102 9999999999101 9999999999100 9999999999099 9999999999098 9999999999097 9999999999096 9999999999095 9999999999094 9999999999093 9999999999092 9999999999091 9999999999090 9999999999089 9999999999088 9999999999087 9999999999086 9999999999085 9999999999084 9999999999083 9999999999082 9999999999081 9999999999080 9999999999079 9999999999078 9999999999077 9999999999076 9999999999075 9999999999074 9999999999073 9999999999072 9999999999071 9999999999070 9999999999069 9999999999068 9999999999067 9999999999066 9999999999065 9999999999064 9999999999063 9999999999062 9999999999061 9999999999060 9999999999059 9999999999058 9999999999057 9999999999056 9999999999055 9999999999054 9999999999053 9999999999052 9999999999051 9999999999050 9999999999049 9999999999048 9999999999047 9999999999046 9999999999045 9999999999044 9999999999043 9999999999042 9999999999041 9999999999040 9999999999039 9999999999038 9999999999037 9999999999036 9999999999035 9999999999034 9999999999033 9999999999032 9999999999031 9999999999030 9999999999029 9999999999028 9999999999027 9999999999026 9999999999025 9999999999024 9999999999023 9999999999022 9999999999021 9999999999020 9999999999019 9999999999018 9999999999017 9999999999016 9999999999015 9999999999014 9999999999013 9999999999012 9999999999011 9999999999010 9999999999009 9999999999008 9999999999007 9999999999006 9999999999005 9999999999004 9999999999003 9999999999002 9999999999001 9999999999000 9999999998999 9999999998998 9999999998997 9999999998996 9999999998995 9999999998994 9999999998993 9999999998992 9999999998991 9999999998990 9999999998989 9999999998988 9999999998987 9999999998986 9999999998985 9999999998984 9999999998983 9999999998982 9999999998981 9999999998980 9999999998979 9999999998978 9999999998977 9999999998976 9999999998975 9999999998974 9999999998973 9999999998972 9999999998971 9999999998970 9999999998969 9999999998968 9999999998967 9999999998966 9999999998965 9999999998964 9999999998963 9999999998962 9999999998961 9999999998960 9999999998959 9999999998958 9999999998957 9999999998956 9999999998955 9999999998954 9999999998953 9999999998952 9999999998951 9999999998950 9999999998949 9999999998948 9999999998947 9999999998946 9999999998945 9999999998944 9999999998943 9999999998942 9999999998941 9999999998940 9999999998939 9999999998938 9999999998937 9999999998936 9999999998935 9999999998934 9999999998933 9999999998932 9999999998931 9999999998930 9999999998929 9999999998928 9999999998927 9999999998926 9999999998925 9999999998924 9999999998923 9999999998922 9999999998921 9999999998920 9999999998919 9999999998918 9999999998917 9999999998916 9999999998915 9999999998914 9999999998913 9999999998912 9999999998911 9999999998910 9999999998909 9999999998908 9999999998907 9999999998906 9999999998905 9999999998904 9999999998903 9999999998902 9999999998901 9999999998900 9999999998899 9999999998898 9999999998897 9999999998896 9999999998895 9999999998894 9999999998893 9999999998892 9999999998891 9999999998890 9999999998889 9999999998888 9999999998887 9999999998886 9999999998885 9999999998884 9999999998883 9999999998882 9999999998881 9999999998880 9999999998879 9999999998878 9999999998877 9999999998876 9999999998875 9999999998874 9999999998873 9999999998872 9999999998871 9999999998870 9999999998869 9999999998868 9999999998867 9999999998866 9999999998865 9999999998864 9999999998863 9999999998862 9999999998861 9999999998860 9999999998859 9999999998858 9999999998857 9999999998856 9999999998855 9999999998854 9999999998853 9999999998852 9999999998851 9999999998850 9999999998849 9999999998848 9999999998847 9999999998846 9999999998845 9999999998844 9999999998843 9999999998842 9999999998841 9999999998840 9999999998839 9999999998838 9999999998837 9999999998836 9999999998835 9999999998834 9999999998833 9999999998832 9999999998831 9999999998830 9999999998829 9999999998828 9999999998827 9999999998826 9999999998825 9999999998824 9999999998823 9999999998822 9999999998821 9999999998820 9999999998819 9999999998818 9999999998817 9999999998816 9999999998815 9999999998814 9999999998813 9999999998812 9999999998811 9999999998810 9999999998809 9999999998808 9999999998807 9999999998806 9999999998805 9999999998804 9999999998803 9999999998802 9999999998801 9999999998800 9999999998799 9999999998798 9999999998797 9999999998796 9999999998795 9999999998794 9999999998793 9999999998792 9999999998791 9999999998790 9999999998789 9999999998788 9999999998787 9999999998786 9999999998785 9999999998784 9999999998783 9999999998782 9999999998781 9999999998780 9999999998779 9999999998778 9999999998777 9999999998776 9999999998775 9999999998774 9999999998773 9999999998772 9999999998771 9999999998770 9999999998769 9999999998768 9999999998767 9999999998766 9999999998765 9999999998764 9999999998763 9999999998762 9999999998761 9999999998760 9999999998759 9999999998758 9999999998757 9999999998756 9999999998755 9999999998754 9999999998753 9999999998752 9999999998751 9999999998750 9999999998749 9999999998748 9999999998747 9999999998746 9999999998745 9999999998744 9999999998743 9999999998742 9999999998741 9999999998740 9999999998739 9999999998738 9999999998737 9999999998736 9999999998735 9999999998734 9999999998733 9999999998732 9999999998731 9999999998730 9999999998729 9999999998728 9999999998727 9999999998726 9999999998725 9999999998724 9999999998723 9999999998722 9999999998721 9999999998720 9999999998719 9999999998718 9999999998717 9999999998716 9999999998715 9999999998714 9999999998713 9999999998712 9999999998711 9999999998710 9999999998709 9999999998708 9999999998707 9999999998706 9999999998705 9999999998704 9999999998703 9999999998702 9999999998701 9999999998700 9999999998699 9999999998698 9999999998697 9999999998696 9999999998695 9999999998694 9999999998693 9999999998692 9999999998691 9999999998690 9999999998689 9999999998688 9999999998687 9999999998686 9999999998685 9999999998684 9999999998683 9999999998682 9999999998681 9999999998680 9999999998679 9999999998678 9999999998677 9999999998676 9999999998675 9999999998674 9999999998673 9999999998672 9999999998671 9999999998670 9999999998669 9999999998668 9999999998667 9999999998666 9999999998665 9999999998664 9999999998663 9999999998662 9999999998661 9999999998660 9999999998659 9999999998658 9999999998657 9999999998656 9999999998655 9999999998654 9999999998653 9999999998652 9999999998651 9999999998650 9999999998649 9999999998648 9999999998647 9999999998646 9999999998645 9999999998644 9999999998643 9999999998642 9999999998641 9999999998640 9999999998639 9999999998638 9999999998637 9999999998636 9999999998635 9999999998634 9999999998633 9999999998632 9999999998631 9999999998630 9999999998629 9999999998628 9999999998627 9999999998626 9999999998625 9999999998624 9999999998623 9999999998622 9999999998621 9999999998620 9999999998619 9999999998618 9999999998617 9999999998616 9999999998615 9999999998614 9999999998613 9999999998612 9999999998611 9999999998610 9999999998609 9999999998608 9999999998607 9999999998606 9999999998605 9999999998604 9999999998603 9999999998602 9999999998601 9999999998600 9999999998599 9999999998598 9999999998597 9999999998596 9999999998595 9999999998594 9999999998593 9999999998592 9999999998591 9999999998590 9999999998589 9999999998588 9999999998587 9999999998586 9999999998585 9999999998584 9999999998583 9999999998582 9999999998581 9999999998580 9999999998579 9999999998578 9999999998577 9999999998576 9999999998575 9999999998574 9999999998573 9999999998572 9999999998571 9999999998570 9999999998569 9999999998568 9999999998567 9999999998566 9999999998565 9999999998564 9999999998563 9999999998562 9999999998561 9999999998560 9999999998559 9999999998558 9999999998557 9999999998556 9999999998555 9999999998554 9999999998553 9999999998552 9999999998551 9999999998550 9999999998549 9999999998548 9999999998547 9999999998546 9999999998545 9999999998544 9999999998543 9999999998542 9999999998541 9999999998540 9999999998539 9999999998538 9999999998537 9999999998536 9999999998535 9999999998534 9999999998533 9999999998532 9999999998531 9999999998530 9999999998529 9999999998528 9999999998527 9999999998526 9999999998525 9999999998524 9999999998523 9999999998522 9999999998521 9999999998520 9999999998519 9999999998518 9999999998517 9999999998516 9999999998515 9999999998514 9999999998513 9999999998512 9999999998511 9999999998510 9999999998509 9999999998508 9999999998507 9999999998506 9999999998505 9999999998504 9999999998503 9999999998502 9999999998501 9999999998500 9999999998499 9999999998498 9999999998497 9999999998496 9999999998495 9999999998494 9999999998493 9999999998492 9999999998491 9999999998490 9999999998489 9999999998488 9999999998487 9999999998486 9999999998485 9999999998484 9999999998483 9999999998482 9999999998481 9999999998480 9999999998479 9999999998478 9999999998477 9999999998476 9999999998475 9999999998474 9999999998473 9999999998472 9999999998471 9999999998470 9999999998469 9999999998468 9999999998467 9999999998466 9999999998465 9999999998464 9999999998463 9999999998462 9999999998461 9999999998460 9999999998459 9999999998458 9999999998457 9999999998456 9999999998455 9999999998454 9999999998453 9999999998452 9999999998451 9999999998450 9999999998449 9999999998448 9999999998447 9999999998446 9999999998445 9999999998444 9999999998443 9999999998442 9999999998441 9999999998440 9999999998439 9999999998438 9999999998437 9999999998436 9999999998435 9999999998434 9999999998433 9999999998432 9999999998431 9999999998430 9999999998429 9999999998428 9999999998427 9999999998426 9999999998425 9999999998424 9999999998423 9999999998422 9999999998421 9999999998420 9999999998419 9999999998418 9999999998417 9999999998416 9999999998415 9999999998414 9999999998413 9999999998412 9999999998411 9999999998410 9999999998409 9999999998408 9999999998407 9999999998406 9999999998405 9999999998404 9999999998403 9999999998402 9999999998401 9999999998400 9999999998399 9999999998398 9999999998397 9999999998396 9999999998395 9999999998394 9999999998393 9999999998392 9999999998391 9999999998390 9999999998389 9999999998388 9999999998387 9999999998386 9999999998385 9999999998384 9999999998383 9999999998382 9999999998381 9999999998380 9999999998379 9999999998378 9999999998377 9999999998376 9999999998375 9999999998374 9999999998373 9999999998372 9999999998371 9999999998370 9999999998369 9999999998368 9999999998367 9999999998366 9999999998365 9999999998364 9999999998363 9999999998362 9999999998361 9999999998360 9999999998359 9999999998358 9999999998357 9999999998356 9999999998355 9999999998354 9999999998353 9999999998352 9999999998351 9999999998350 9999999998349 9999999998348 9999999998347 9999999998346 9999999998345 9999999998344 9999999998343 9999999998342 9999999998341 9999999998340 9999999998339 9999999998338 9999999998337 9999999998336 9999999998335 9999999998334 9999999998333 9999999998332 9999999998331 9999999998330 9999999998329 9999999998328 9999999998327 9999999998326 9999999998325 9999999998324 9999999998323 9999999998322 9999999998321 9999999998320 9999999998319 9999999998318 9999999998317 9999999998316 9999999998315 9999999998314 9999999998313 9999999998312 9999999998311 9999999998310 9999999998309 9999999998308 9999999998307 9999999998306 9999999998305 9999999998304 9999999998303 9999999998302 9999999998301 9999999998300 9999999998299 9999999998298 9999999998297 9999999998296 9999999998295 9999999998294 9999999998293 9999999998292 9999999998291 9999999998290 9999999998289 9999999998288 9999999998287 9999999998286 9999999998285 9999999998284 9999999998283 9999999998282 9999999998281 9999999998280 9999999998279 9999999998278 9999999998277 9999999998276 9999999998275 9999999998274 9999999998273 9999999998272 9999999998271 9999999998270 9999999998269 9999999998268 9999999998267 9999999998266 9999999998265 9999999998264 9999999998263 9999999998262 9999999998261 9999999998260 9999999998259 9999999998258 9999999998257 9999999998256 9999999998255 9999999998254 9999999998253 9999999998252 9999999998251 9999999998250 9999999998249 9999999998248 9999999998247 9999999998246 9999999998245 9999999998244 9999999998243 9999999998242 9999999998241 9999999998240 9999999998239 9999999998238 9999999998237 9999999998236 9999999998235 9999999998234 9999999998233 9999999998232 9999999998231 9999999998230 9999999998229 9999999998228 9999999998227 9999999998226 9999999998225 9999999998224 9999999998223 9999999998222 9999999998221 9999999998220 9999999998219 9999999998218 9999999998217 9999999998216 9999999998215 9999999998214 9999999998213 9999999998212 9999999998211 9999999998210 9999999998209 9999999998208 9999999998207 9999999998206 9999999998205 9999999998204 9999999998203 9999999998202 9999999998201 9999999998200 9999999998199 9999999998198 9999999998197 9999999998196 9999999998195 9999999998194 9999999998193 9999999998192 9999999998191 9999999998190 9999999998189 9999999998188 9999999998187 9999999998186 9999999998185 9999999998184 9999999998183 9999999998182 9999999998181 9999999998180 9999999998179 9999999998178 9999999998177 9999999998176 9999999998175 9999999998174 9999999998173 9999999998172 9999999998171 9999999998170 9999999998169 9999999998168 9999999998167 9999999998166 9999999998165 9999999998164 9999999998163 9999999998162 9999999998161 9999999998160 9999999998159 9999999998158 9999999998157 9999999998156 9999999998155 9999999998154 9999999998153 9999999998152 9999999998151 9999999998150 9999999998149 9999999998148 9999999998147 9999999998146 9999999998145 9999999998144 9999999998143 9999999998142 9999999998141 9999999998140 9999999998139 9999999998138 9999999998137 9999999998136 9999999998135 9999999998134 9999999998133 9999999998132 9999999998131 9999999998130 9999999998129 9999999998128 9999999998127 9999999998126 9999999998125 9999999998124 9999999998123 9999999998122 9999999998121 9999999998120 9999999998119 9999999998118 9999999998117 9999999998116 9999999998115 9999999998114 9999999998113 9999999998112 9999999998111 9999999998110 9999999998109 9999999998108 9999999998107 9999999998106 9999999998105 9999999998104 9999999998103 9999999998102 9999999998101 9999999998100 9999999998099 9999999998098 9999999998097 9999999998096 9999999998095 9999999998094 9999999998093 9999999998092 9999999998091 9999999998090 9999999998089 9999999998088 9999999998087 9999999998086 9999999998085 9999999998084 9999999998083 9999999998082 9999999998081 9999999998080 9999999998079 9999999998078 9999999998077 9999999998076 9999999998075 9999999998074 9999999998073 9999999998072 9999999998071 9999999998070 9999999998069 9999999998068 9999999998067 9999999998066 9999999998065 9999999998064 9999999998063 9999999998062 9999999998061 9999999998060 9999999998059 9999999998058 9999999998057 9999999998056 9999999998055 9999999998054 9999999998053 9999999998052 9999999998051 9999999998050 9999999998049 9999999998048 9999999998047 9999999998046 9999999998045 9999999998044 9999999998043 9999999998042 9999999998041 9999999998040 9999999998039 9999999998038 9999999998037 9999999998036 9999999998035 9999999998034 9999999998033 9999999998032 9999999998031 9999999998030 9999999998029 9999999998028 9999999998027 9999999998026 9999999998025 9999999998024 9999999998023 9999999998022 9999999998021 9999999998020 9999999998019 9999999998018 9999999998017 9999999998016 9999999998015 9999999998014 9999999998013 9999999998012 9999999998011 9999999998010 9999999998009 9999999998008 9999999998007 9999999998006 9999999998005 9999999998004 9999999998003 9999999998002 9999999998001 9999999998000 9999999997999 9999999997998 9999999997997 9999999997996 9999999997995 9999999997994 9999999997993 9999999997992 9999999997991 9999999997990 9999999997989 9999999997988 9999999997987 9999999997986 9999999997985 9999999997984 9999999997983 9999999997982 9999999997981 9999999997980 9999999997979 9999999997978 9999999997977 9999999997976 9999999997975 9999999997974 9999999997973 9999999997972 9999999997971 9999999997970 9999999997969 9999999997968 9999999997967 9999999997966 9999999997965 9999999997964 9999999997963 9999999997962 9999999997961 9999999997960 9999999997959 9999999997958 9999999997957 9999999997956 9999999997955 9999999997954 9999999997953 9999999997952 9999999997951 9999999997950 9999999997949 9999999997948 9999999997947 9999999997946 9999999997945 9999999997944 9999999997943 9999999997942 9999999997941 9999999997940 9999999997939 9999999997938 9999999997937 9999999997936 9999999997935 9999999997934 9999999997933 9999999997932 9999999997931 9999999997930 9999999997929 9999999997928 9999999997927 9999999997926 9999999997925 9999999997924 9999999997923 9999999997922 9999999997921 9999999997920 9999999997919 9999999997918 9999999997917 9999999997916 9999999997915 9999999997914 9999999997913 9999999997912 9999999997911 9999999997910 9999999997909 9999999997908 9999999997907 9999999997906 9999999997905 9999999997904 9999999997903 9999999997902 9999999997901 9999999997900 9999999997899 9999999997898 9999999997897 9999999997896 9999999997895 9999999997894 9999999997893 9999999997892 9999999997891 9999999997890 9999999997889 9999999997888 9999999997887 9999999997886 9999999997885 9999999997884 9999999997883 9999999997882 9999999997881 9999999997880 9999999997879 9999999997878 9999999997877 9999999997876 9999999997875 9999999997874 9999999997873 9999999997872 9999999997871 9999999997870 9999999997869 9999999997868 9999999997867 9999999997866 9999999997865 9999999997864 9999999997863 9999999997862 9999999997861 9999999997860 9999999997859 9999999997858 9999999997857 9999999997856 9999999997855 9999999997854 9999999997853 9999999997852 9999999997851 9999999997850 9999999997849 9999999997848 9999999997847 9999999997846 9999999997845 9999999997844 9999999997843 9999999997842 9999999997841 9999999997840 9999999997839 9999999997838 9999999997837 9999999997836 9999999997835 9999999997834 9999999997833 9999999997832 9999999997831 9999999997830 9999999997829 9999999997828 9999999997827 9999999997826 9999999997825 9999999997824 9999999997823 9999999997822 9999999997821 9999999997820 9999999997819 9999999997818 9999999997817 9999999997816 9999999997815 9999999997814 9999999997813 9999999997812 9999999997811 9999999997810 9999999997809 9999999997808 9999999997807 9999999997806 9999999997805 9999999997804 9999999997803 9999999997802 9999999997801 9999999997800 9999999997799 9999999997798 9999999997797 9999999997796 9999999997795 9999999997794 9999999997793 9999999997792 9999999997791 9999999997790 9999999997789 9999999997788 9999999997787 9999999997786 9999999997785 9999999997784 9999999997783 9999999997782 9999999997781 9999999997780 9999999997779 9999999997778 9999999997777 9999999997776 9999999997775 9999999997774 9999999997773 9999999997772 9999999997771 9999999997770 9999999997769 9999999997768 9999999997767 9999999997766 9999999997765 9999999997764 9999999997763 9999999997762 9999999997761 9999999997760 9999999997759 9999999997758 9999999997757 9999999997756 9999999997755 9999999997754 9999999997753 9999999997752 9999999997751 9999999997750 9999999997749 9999999997748 9999999997747 9999999997746 9999999997745 9999999997744 9999999997743 9999999997742 9999999997741 9999999997740 9999999997739 9999999997738 9999999997737 9999999997736 9999999997735 9999999997734 9999999997733 9999999997732 9999999997731 9999999997730 9999999997729 9999999997728 9999999997727 9999999997726 9999999997725 9999999997724 9999999997723 9999999997722 9999999997721 9999999997720 9999999997719 9999999997718 9999999997717 9999999997716 9999999997715 9999999997714 9999999997713 9999999997712 9999999997711 9999999997710 9999999997709 9999999997708 9999999997707 9999999997706 9999999997705 9999999997704 9999999997703 9999999997702 9999999997701 9999999997700 9999999997699 9999999997698 9999999997697 9999999997696 9999999997695 9999999997694 9999999997693 9999999997692 9999999997691 9999999997690 9999999997689 9999999997688 9999999997687 9999999997686 9999999997685 9999999997684 9999999997683 9999999997682 9999999997681 9999999997680 9999999997679 9999999997678 9999999997677 9999999997676 9999999997675 9999999997674 9999999997673 9999999997672 9999999997671 9999999997670 9999999997669 9999999997668 9999999997667 9999999997666 9999999997665 9999999997664 9999999997663 9999999997662 9999999997661 99999999{-truncated-}
You are reusing the same array pointer. But not really re using the exact same memory space. Its not really the best practice in the case you mentioned.
You can create the array (alloccate memory) only once in the beginning of your code and re use it in all further cases.
import java.io.*;
import java.util.*;
import java.util.stream.*;
public class Solution {
public static void main(String[] args) {
/* Enter your code here. Read input from STDIN. Print output to STDOUT. Your class should be named Solution. */
Scanner sc = new Scanner(System.in);
int t = sc.nextInt();
int b;
long n;
long k;
long []array;
array = new long [100005]; // biggest size that can be possible
int count;
long sum = 0;
for(int i = 0; i < t; i++){
n = sc.nextLong();
k = sc.nextLong();
b = sc.nextInt();
count = 0;
sum = 0;
//no need to create it again and again, just create it once but create it as the biggest size
for (int j = 0; j < b; j++){
array[b - 1 - j]=j+1;
sum +=(j+1);
}
while (true){
while (sum < n && count < b && k > b){
if ((sum + (k - count - array[count]) )< n) {
sum += k - count - array[count];
array[count]+=k - count - array[count];
}
else {
System.out.println (n - sum);
array[count]+= n - sum;
sum += n - sum;
}
if (array[count] == k - count) count++;
}
if (sum == n) {
for (int x = 0; x < b; x++){
System.out.print (array[x]);
if (x < b - 1) System.out.print(" ");
else System.out.print("\n");
}
break;
}
else {
System.out.println ("-1");
break;
}
}
}
}
}
I am using batch processing to write into InfluxDB and below is my code for doing that.
String dbName = "test";
influxDB.query(new Query("CREATE DATABASE " + dbName, dbName));
Stopwatch watch = Stopwatch.createStarted();
influxDB.enableBatch(2000, 100, TimeUnit.MILLISECONDS);
for (int j = 0; j < 100000; j++) {
Point point = Point.measurement("cpu")
.addField("idle", (double) j)
.addField("system", 3.0 * j).build();
influxDB.write(dbName, "autogen", point);
}
influxDB.disableBatch();
System.out.println("Write for " + 100000 + " Points took:" + watch);
}
Here i am writing 100000 points and which is taking very reasonable time to write, however only few records are written into DB instead of expected 100000 records.
select count(idle) from cpu gives me only "89" i am expecting it to be "100000"
While select * from cpu gives me following:
cpu
time idle system
2016-10-06T23:57:41.184Z 8 24
2016-10-06T23:57:41.185Z 196 588
2016-10-06T23:57:41.186Z 436 1308
2016-10-06T23:57:41.187Z 660 1980
2016-10-06T23:57:41.188Z 916 2748
2016-10-06T23:57:41.189Z 1278 3834
2016-10-06T23:57:41.19Z 1405 4215
2016-10-06T23:57:41.191Z 1409 4227
2016-10-06T23:57:41.192Z 1802 5406
2016-10-06T23:57:41.193Z 1999 5997
2016-10-06T23:57:41.456Z 3757 11271
2016-10-06T23:57:41.457Z 3999 11997
2016-10-06T23:57:41.858Z 4826 14478 and so on.....
Here my question is why the values of idle are missing, for example, after 8 it should 9, 10, 11, and so on but these values were not persisted and comes directly 196 and then missing in between and then 436. Any idea how to persist all value of loop variable "j" in this situation?
This line
influxDB.enableBatch(2000, 100, TimeUnit.MILLISECONDS);
says that it will flush input data if there are more than 2000 samples per 100 ms period. Since you are trying to write 100k samples then logically most of them get flushed.
Instead, write less samples in a single batch. My recommendation would be to write 5000 samples in a single batch, and make multiple batches until all your data is in the db.
// Batch 1
influxDB.enableBatch(5000, 100, TimeUnit.MILLISECONDS);
for (int j = 0; j < 5000; j++) {
Point point = Point.measurement("cpu")
.addField("idle", (double) j)
.addField("system", 3.0 * j).build();
influxDB.write(dbName, "autogen", point);
}
influxDB.disableBatch();
// Batch 2
// ...
My project at work is using the Jackson JSON serializer to convert a bunch of Java objects into Strings in order to send them to REST services.
Some of these objects contain sensitive data, so I've written custom serializers to serialize these objects to JSON strings, then gzip them, then encrypt them using AES;
This turns the strings into byte arrays, so I use the Base64 encoder in Apache commons codec to convert the byte arrays into strings. The custom deserializers behind the REST interfaces reverse this process:
base64 decode -> decrypt -> decompress -> deserialize using default Jackson deserializer.
Base64 encoding increases the size of the output (the gzip step in serialization is meant to help ameliorate this increase), so I checked Google to see if there was a more efficient alternative, which led me to this previous stackoverflow thread that brought up Ascii85 encoding as a more efficient alternative -
Base64 adds 33% to the size of the output, Ascii85 adds 25% to the size of the output.
I found a few Java Ascii85 implementations e.g. Apache pdfbox, but I'm a bit leery to use the encoding - it seems like hardly anybody is using or implementing it, which might just mean that Base64 has more inertia, or which may instead mean that there's some wonky problem with Ascii85.
Does anybody know more on this subject? Are there any problems with Ascii85 that mean that I should use Base64 instead?
Base64 is way more common. The difference in size really isn't that significant in most cases, and if you add at the HTTP level (which will compress the base64) instead of within your payload, you may well find the difference goes away entirely.
Are there any problems with Ascii85 that mean that I should use Base64 instead?
I would strongly advise using base64 just because it's so much more widespread. It's pretty much the canonical way of representing binary data as text (unless you want to use hex, of course).
ASCII85 is a nice encoding to use to save that extra bit of space. But it outputs many characters that would need to be escaped if naively sent over HTTP. Base64 encoding has a variant that can be sent over HTTP without any escaping.
Here's a javascript ASCII85 encoder in case anyone needs to try:
// By Steve Hanov. Released to the public domain.
function encodeAscii85(input) {
var output = "<~";
var chr1, chr2, chr3, chr4, chr, enc1, enc2, enc3, enc4, enc5;
var i = 0;
while (i < input.length) {
// Access past the end of the string is intentional.
chr1 = input.charCodeAt(i++);
chr2 = input.charCodeAt(i++);
chr3 = input.charCodeAt(i++);
chr4 = input.charCodeAt(i++);
chr = ((chr1 << 24) | (chr2 << 16) | (chr3 << 8) | chr4) >>> 0;
enc1 = (chr / (85 * 85 * 85 * 85) | 0) % 85 + 33;
enc2 = (chr / (85 * 85 * 85) | 0) % 85 + 33;
enc3 = (chr / (85 * 85) | 0 ) % 85 + 33;
enc4 = (chr / 85 | 0) % 85 + 33;
enc5 = chr % 85 + 33;
output += String.fromCharCode(enc1) +
String.fromCharCode(enc2);
if (!isNaN(chr2)) {
output += String.fromCharCode(enc3);
if (!isNaN(chr3)) {
output += String.fromCharCode(enc4);
if (!isNaN(chr4)) {
output += String.fromCharCode(enc5);
}
}
}
}
output += "~>";
return output;
}
<input onKeyUp="result.innerHTML = encodeAscii85(this.value)" placeholder="write text here" type="text">
<p id="result"></p>
Here is matching ASCII85 AKA Base85 decoder (for user Qwerty) in JavaScript:
function decode_ascii85(a) {
var c, d, e, f, g, h = String, l = "length", w = 255, x = "charCodeAt", y = "slice", z = "replace";
for ("<~" === a[y](0, 2) && "~>" === a[y](-2), a = a[y](2, -2)[z](/\s/g, "")[z]("z", "!!!!!"),
c = "uuuuu"[y](a[l] % 5 || 5), a += c, e = [], f = 0, g = a[l]; g > f; f += 5) d = 52200625 * (a[x](f) - 33) + 614125 * (a[x](f + 1) - 33) + 7225 * (a[x](f + 2) - 33) + 85 * (a[x](f + 3) - 33) + (a[x](f + 4) - 33),
e.push(w & d >> 24, w & d >> 16, w & d >> 8, w & d);
return function(a, b) {
for (var c = b; c > 0; c--) a.pop();
}(e, c[l]), h.fromCharCode.apply(h, e);
}
<input onKeyUp="result.innerHTML = decode_ascii85(this.value)" placeholder="insert encoded string here" type="text">
<p id="result"></p>
example: <xmp><~<+oue+DGm>#3BW*D/a<&+EV19F<L~></xmp>
I run a small online gaming community and deal with a database of accounts.
The setup is this:
Folder named Accounts
Inside the Accounts directory, there is 200,000+ text files organized by player name. Access to this folder manually is a pain because of the needed RAM to get in and search files. I find this very inconvenient.
I access this directory to send password reminders or for highscores on who has been playing the longest.
Here is an example of an account file. This file is named Falcon.txt
[ACCOUNT]
character-username = Falcon
character-password = falconpassword
[INFO]
character-coordx = 3252
character-coordy = 3432
character-active = yes
character-ismember = 1
character-messages = 5
character-lastconnection = [removed]
character-lastlogin = 2009-11-29
character-energy = 100
character-gametime = 193
character-gamecount = 183
[EQUIPMENT]
character-equip = 0 4724 0
character-equip = 1 1052 0
character-equip = 2 6585 0
character-equip = 3 4151 0
character-equip = 4 4720 0
character-equip = 5 1215 0
character-equip = 6 -1 0
character-equip = 7 4722 0
character-equip = 8 -1 0
character-equip = 9 775 0
character-equip = 10 1837 0
character-equip = 11 -1 0
character-equip = 12 6735 0
character-equip = 13 -1 0
[APPEARANCE]
character-look = 0 1
character-look = 1 1
character-look = 2 2
character-look = 3 3
character-look = 4 5
character-look = 5 2
[STATS]
character-skill = 0 1 0
character-skill = 1 1 0
character-skill = 2 1 0
character-skill = 3 1 0
character-skill = 4 1 0
character-skill = 5 1 0
character-skill = 6 1 0
character-skill = 7 1 0
character-skill = 8 1 0
character-skill = 9 1 0
character-skill = 10 1 0
character-skill = 11 1 0
character-skill = 12 1 0
character-skill = 13 1 0
character-skill = 14 1 0
character-skill = 15 1 0
character-skill = 16 1 0
character-skill = 17 1 0
character-skill = 18 1 0
character-skill = 19 1 0
character-skill = 20 1 0
[ITEMS]
[BANK]
[FRIENDS]
[IGNORES]
[END]
There is a huge database of these and search through the directory in the files for values.
Values I mean by item ID's or IP addresses to find and track other accounts.
However I have a new problem and my development for this is crashing.
As you can see in the file the lines are organized by tabs.
character-equip = 0 4724 1
If I put the value 4724 in my search application, I want it to print out the value 1 tab to the right of the found search result. I want it to print out the value for the found results only, not extra results.
So the search could look like this:
1 "Enter item to find:"
2 "Enter item to find: 4724"
3 "Account Falcon.txt has 1!"
press any key to continue...
Or if there was more quantity of that equipped item
character-equip = 5 1239 102
1. "Enter item to find:"
2. "Enter item to find: 1239"
3. "Account Falcon2.txt has 102!"
press any key to continue...
I simply want to input an item ID, and have it display the value after the found value. The white space is a tab. I have tried doing this and the only successful way of getting any result is to put a tab in between the search term. So if I want to find item 1239 id type this in the cmd line:
Enter item to find:<tab>1239<tab>
It would then search for that and will display the accounts with that item in it. However I still have to individually open up the accounts to find out the quantity of that item. I want the search results to display the quantity of the item if the value is found. However if the value is a quantity and it trys to search one tab over, I want it to either skip it or say zero.
This is what I mean.
character-equip = 0 1024 1239
Enter item to find: 1239
If it hits this account I want to make the search results display a zero if it cannot tab over or view any values in the white space. So it will display as null or zero
Account Falcon3.txt has null!
or
Account Falcon3.txt has 0!
I've attempted to do this but I am unsure how to achieve this.
Here is my code.
import java.io.*;
import java.util.*;
public class ItemDatabase {
public static void main(String args[]) {
System.out.print("Enter item to find: ");
Scanner sc = new Scanner(System.in);
find(sc.nextLine());
}
public static void find(String delim) {
File dir = new File("accounts");
if (dir.exists()) {
String read;
try {
File files[] = dir.listFiles();
for (int i = 0; i < files.length; i++) {
File loaded = files[i];
if (loaded.getName().endsWith(".txt")) {
BufferedReader in = new BufferedReader(new FileReader(loaded));
StringBuffer load = new StringBuffer();
while ((read = in.readLine()) != null) {
load.append(read + "\n");
}
String delimiter[] = new String(load).split(delim);
if(delimiter.length > 1) {
System.out.println("Account " + loaded.getName() + "has " + delimiter[1] + "!");
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
} else {
System.out.println("error: dir wasn't found!");
}
}
}
Thanks guys I hope you can help me.
This is simply crying out for a database. If your entire back end is running in a single java process I'd recommend going with something like Apache Derby or H2.
If you'd rather not move to a database, and the main problem is the act of listing all the entries in a single directory, could you split it into a heirarchy. Your Falcon account could then be located in F/FA/Falcon.txt - each directory would then contain a more manageable number of files.
Aside from the need for a database, you could probably implement your solution more intuitively and easily using commandline utilities such as find, grep, etc. or a text-processing language such as perl.
Maybe
grep '[0-9]+\t[0-9]+\t1239' Account_Falcon3.txt
Would return
character-equip = 0 1024 1239
You could then easily see that the value is 0 for that item.
I cannot emphasize enough the need to not write a Java program to do this - you won't do as good a job as the authors of the standard shell utilities. Let's face it, the fact that you are asking this question indicates that you are a newb! :) (We are all newbs depending on the topic).