We Keep Coding

String Index Exception

I was trying to convert an expression from infix form to postfix form.I used String a, p , s for stack, postfix result expression, input expression respectively.
Every time I am getting this error:
Exception in thread "main" java.lang.StringIndexOutOfBoundsException:
String index out of range: -1 at
java.lang.String.charAt(String.java:658) at
javaapplication4.A.conversion(A.java:50) at
javaapplication4.A.main(A.java:83)
Please help me how can I solve it.
Here is my code:
import java.util.Scanner;
public class A {
String a="(", s = "", p = "";
int i, n = 1, top = 0, pp = 0;
void push(char ch) {
a = a + ch;
top = n;
n++;
}
void pop() {
n--;
top--;
}
int prio(char ch) {
int f = -1;
if (ch == '(') {
f = 0;
} else if (ch == '+' || ch == '-') {
f = 1;
} else if (ch == '*' || ch == '/' || ch == '%') {
f = 2;
} else if (ch == '^') {
f = 3;
}
return f;
}
void conversion() {
System.out.print("Enter infix form: ");
Scanner sd = new Scanner(System.in);
s = sd.nextLine();
//System.out.println(s);
int t, j, sz;
sz = s.length();
for (i = 0; i < sz; i++) {
if (s.charAt(i) >= '0' && s.charAt(i) <= '9') {
p = p + s.charAt(i);
pp++;
} else if (s.charAt(i) == '(') {
push('(');
} else if (s.charAt(i) == '-' || s.charAt(i) == '+' || s.charAt(i) == '*' || s.charAt(i) == '/' || s.charAt(i) == '%' || s.charAt(i) == '^') {
j = prio(s.charAt(i));
t = prio(a.charAt(top));
//System.out.println(t+" "+j);
while (j <= t) {
p = p + a.charAt(top);
pp++;
pop();
t = prio(a.charAt(top));
}
push(s.charAt(i));
} else if (s.charAt(i) == ')') {
while (a.charAt(top) != '(') {
p = p + a.charAt(top);
pp++;
pop();
}
pop();
}
}
while (a.charAt(top) != '(') {
p = p + a.charAt(top);
pp++;
pop();
}
pop();
}
void postfix() {
System.out.print("postfix form is: ");
System.out.println(p);
}
public static void main(String args[]) {
A h = new A();
h.conversion();
h.postfix();
//System.out.println(h.a);
//System.out.println(h.s);
}
}

Can you confirm if the error is here?
while (a.charAt(top) != '(')
Inside the loop you continuously pop(), which decrements from top, which has the risk of going negative if a match is never found. Even if the error is not here, it should check for that condition.

You may have made an extra pop() definition. Can you check this?

FileWriter not writing to output file

I can run this code just printing to the console and everything prints fine, and then when I change all the System.out.println's to outputStream.write
("some string") nothing writes to the output file at all. It is just blank. The first file is the main file. Then my other classes are MyStack and PostfixError. I'll include them below this one. The code takes in input from an input file entered in the command line and then outputs it to a file also specified by the command line. The input file is supposed to be in Postfix format such as ABC+- or AB+C. If it is not in that format it will produce an error. If the input is in that format the code will take that input and write the instructions that would be needed to evaluate the postfix expression if we were using a computer with only one register.
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.io.FileWriter;
public class PostfixEvaluation{
public static void main(String[] args) throws PostfixError, IOException{
FileReader inputStream = null;
FileReader inputStream2 = null;
FileWriter outputStream = null;
BufferedReader str = null;
int c, j=1,letter;
int stringSize = 100;
String message, letterA, letterB;
try {
inputStream = new FileReader(args[0]); // input file
inputStream2 = new FileReader(args[0]);
outputStream = new FileWriter(args[1]); // output file
str = new BufferedReader(inputStream);
String nextLine; //= str.readLine();
while ((nextLine = str.readLine()) != null) {
outputStream.write("");
outputStream.write("Current expression being evaluated is " + nextLine);
MyStack letterStack = new MyStack(stringSize);
try {
while ((c = inputStream2.read()) != 10){
//letterStack = EvaluateInstructions(c, letterStack,j);
letter = c;
if (letter == 65){ // use an or to account for lower case letters
letterStack.push("A");
}else if (letter == 66) {
letterStack.push("B");
}else if (letter == 67) {
letterStack.push("C");
}else if (letter == 68) {
letterStack.push("D");
}else if (letter == 69) {
letterStack.push("E");
}else if (letter == 70) {
letterStack.push("F");
}else if (letter == 71) {
letterStack.push("G");
}else if (letter == 72) {
letterStack.push("H");
}else if (letter == 73) {
letterStack.push("I");
}else if (letter == 74) {
letterStack.push("J");
}else if (letter == 75) {
letterStack.push("K");
}else if (letter == 76) {
letterStack.push("L");
}else if (letter == 77) {
letterStack.push("M");
}else if (letter == 78) {
letterStack.push("N");
}else if (letter == 79) {
letterStack.push("O");
}else if (letter == 80) {
letterStack.push("P");
}else if (letter == 81) {
letterStack.push("Q");
}else if (letter == 82) {
letterStack.push("R");
}else if ( letter == 83) {
letterStack.push("S");
}else if (letter == 84) {
letterStack.push("T");
}else if (letter == 85) {
letterStack.push("U");
}else if (letter == 86) {
letterStack.push("V");
}else if (letter == 87) {
letterStack.push("W");
}else if (letter == 88){
letterStack.push("X");
}else if (letter == 89) {
letterStack.push("Y");
}else if (letter == 90) {
letterStack.push("Z");
}else if (letter == 42){ //letter == '*'
if (letterStack.isEmpty()|letterStack.length() <= 1) {
j=1;
throw new PostfixError("There are more operators than operands; This is not a valid expression.");
}else {
letterA = letterStack.pop();
letterB = letterStack.pop();
outputStream.write("LD " + letterB);
outputStream.write("ML " + letterA);
outputStream.write("ST TEMP"+j);
letterStack.push("TEMP"+j);
//throw new PrintToOutput("hello");
//j++;
}
}else if (letter == 47) { //letter == '/'
if (letterStack.isEmpty()|letterStack.length() <= 1) {
j=1;
throw new PostfixError("There are more operators than operands; This is not a valid expression.");
}else {
letterA = letterStack.pop();
letterB = letterStack.pop();
outputStream.write("LD " + letterB);
outputStream.write("DV " + letterA);
outputStream.write("ST TEMP"+j);
letterStack.push("TEMP"+j);
//j++;
}
}else if (letter == 43) { //letter == '+'
if (letterStack.isEmpty()|letterStack.length() <= 1) {
j=1;
throw new PostfixError("There are more operators than operands; This is not a valid expression.");
}else {
letterA = letterStack.pop();
letterB = letterStack.pop();
outputStream.write("LD " + letterB);
outputStream.write("AD " + letterA);
outputStream.write("ST TEMP"+j);
letterStack.push("TEMP"+j);
//j++;
}
}else if (letter == 45) { //letter == '-'
if (letterStack.isEmpty()|letterStack.length() <= 1) {
j=1;
throw new PostfixError("There are more operators than operands. This is not a valid expression.");
}else {
letterA = letterStack.pop();
letterB = letterStack.pop();
outputStream.write("LD " + letterB);
outputStream.write("SB " + letterA);
outputStream.write("ST TEMP"+j);
letterStack.push("TEMP"+j);
//j++;
}
}else if (letter == 13) {
//do nothing
}else if (letter == -1) {
outputStream.write("");
outputStream.write("Success! File is finished being read.");
System.exit(0);
}else {
j=1;
//need to empty stack
throw new PostfixError( "This input has an incorrect character or combination of characters.");
}
if (c == 47 | c == 42 | c == 45 |c == 43) {
j++;
}else if (c == 13) {
j=1;
}
}
}catch (PostfixError e) {
outputStream.write(e.getMessage());
//read input stream until it equals 10
do {
c=inputStream2.read();
}while (c !=10);
}
}
} finally {
if (inputStream != null) inputStream.close();
if (inputStream2 !=null) inputStream2.close();
if (outputStream != null) outputStream.close();
str.close();
}
}
}
MyStack Class
public class MyStack {
private String[] theStack;
private int size;
private int top;
private int totalLength;
public MyStack() {
size = 100;
theStack = new String[size];
top = -1;
}
public MyStack(int stringSize) {
size = 100;
theStack = new String[stringSize];
top = 0;
}
public void push(String symbol) {
theStack[top] = symbol;
top++;
}
public String pop() {
return theStack[--top];
}
public boolean isEmpty() {
if (top == -1) {
return true;
}else {
return false;
}
}
public int length() {
return totalLength = top;
}
public void print() {
for(int i=0;i<=top;i++){
System.out.print(theStack[i]+ " ");
}
System.out.println();
}
}
PostfixError Class
public class PostfixError extends Exception{
public PostfixError(String message) {
super(message);
}
}

You call System.exit; this immediately exits; your 'finally' block is NOT run. As a consequence, your BufferedWriter is just abandoned; neither flush() nor close() is invoked on it which means it won't flush its buffer into the file.
SOLUTION: Don't call System.exit there.

how to check number exists between braces

import java.util.Stack;
import java.util.Scanner;
public class CheckValidLocationofParenthensies {
public static void main(String args[]) {
Scanner scanner = new Scanner(System.in);
System.out.print("Enter five data");
String input1 = scanner.next();
balancedParenthensies(input1);
}
public static boolean balancedParenthensies(String s) {
Stack<Character> stack = new Stack<Character>();
for(int i = 0; i < s.length(); i++) {
char c = s.charAt(i);
if(c == '[' || c == '(' || c == '{' ) {
stack.push(c);
if(c == '[') {
newvalueforforward(s,']', i);
}
if(c == '{') {
newvalueforforward(s,'}', i);
}
if(c == '(') {
newvalueforforward(s,')', i);
}
} else if(c == ']') {
if(stack.isEmpty() || stack.pop() != '[') {
newvalue(s,'[', i);
return false;
}
} else if(c == ')') {
if(stack.isEmpty() || stack.pop() != '(') {
newvalue(s,'(', i);
return false;
}
} else if(c == '}') {
if(stack.isEmpty() || stack.pop() != '{') {
newvalue(s,'{', i);
return false;
}
}
}
return stack.isEmpty();
}
public static void newvalueforforward(String userval,char value,int decremntval) {
for(int i = 0; i < userval.length(); i++){
StringBuilder newvalue = new StringBuilder(userval);
int location=i;
newvalue.insert(i, value);
boolean valid= checkingnewvalueisValidorNot(newvalue, location);
location=i+1;
if(valid) {
System.out.println(newvalue+" "+""+location);
}
}
}
public static void newvalue(String userval,char value,int decremntval) {
for(int i = decremntval; i >= 0; i--){
StringBuilder newvalue = new StringBuilder(userval);
int location=decremntval - i;
newvalue.insert(decremntval - i, value);
boolean valid= checkingnewvalueisValidorNot(newvalue, location);
if(valid) {
System.out.println(newvalue+" "+""+location);
}
}
}
public static boolean checkingnewvalueisValidorNot(StringBuilder userval,int validpath) {
Stack<Character> stack = new Stack<Character>();
for(int i = 0; i < userval.length(); i++) {
char c = userval.charAt(i);
if(c == '[' || c == '(' || c == '{' ) {
stack.push(c);
} else if(c == ']') {
if(stack.isEmpty() || stack.pop() != '[') {
return false;
}
} else if(c == ')') {
if(stack.isEmpty() || stack.pop() != '(') {
return false;
}
} else if(c == '}') {
if(stack.isEmpty() || stack.pop() != '{') {
return false;
}
}
}
return stack.isEmpty();
}
}
Above is the code i have written to check whether input string contains all balanced brackets if it is not balanced then get missing bracket and place bracket in all index then again check whether string is balanced or not.
I got valid output but problem is between i bracket there should be a intergers
here is input and outputs
input missing outputs
{[(2+3)*6/10} ] {[](2+3)*6/10} 3 not valid(no numbres btn bracket)
{[(2+3)]*6/10} 8 valid
{[(2+3)*]6/10} 9 not valid(after * no number)
{[(2+3)*6]/10} 10 valid
{[(2+3)*6/]10} 11 not valid( / before bracket)
{[(2+3)*6/1]0} 12 not valid( / btn num bracket)
{[(2+3)*6/10]} 13 valid
i am failing to do proper validation to my output.

Before the bracket there may be:
number
closing bracket
After the bracket there may be:
operator
closing bracket
end of expression
(ignoring any whitespace)

Final return statement, ignore

Im working on a token iterator (valid tokens, "true, false, "true", "&", "!", "(", "false", "^", "true", ")".
The code is working, my question is about return values. I often run into this problem, I have return statements, but the final return statement throws off my result by duplicating the last return statement.
I think I know for sure the error lays within my placement of { and } and while i've learned they aren't necessary, since there's so many nested if's i feel they are necessary.
This seems to be a common problem to me and others ive worked with, does anyone have an idea of how to prevent this problem from happening? Thanks!
My code outputs:
line: [ ! BAD (true ^ false) % truelybad]
next token: [!]
next token: [(]
next token: [true]
next token: [^]
next token: [false]
next token: [)]
next token: [)]
and should output
next token: [!]
next token: [(]
next token: [true]
next token: [^]
next token: [false]
next token: [)]
public class TokenIter implements Iterator<String> {
ArrayList<String> token = new ArrayList<String>();
static int count = 0;
// input line to be tokenized
private String line;
// the next Token, null if no next Token
private String nextToken;
// implement
public TokenIter(String line) {
this.line = line;
}
#Override
// implement
public boolean hasNext() {
// System.out.println(count);
return count < line.length();
}
#Override
// implement
public String next() {
while (hasNext()) {
char c = line.charAt(count);
if (c == '!' || c == '!' || c == '^' || c == '(' || c == ')') {
token.add(Character.toString(c));
count++;
nextToken = Character.toString(c);
return nextToken;
} else if (c == 't' || c == 'T') {
count++;
c = line.charAt(count);
if (c == 'r') {
count++;
c = line.charAt(count);
}
if (c == 'u') {
count++;
c = line.charAt(count);
}
if (c == 'e') {
count++;
c = line.charAt(count);
}if (c == ' ' || c == '!' || c == '!' || c == '^' || c == '(' || c == ')'){
token.add("true");
nextToken = "true";
//count++;
return nextToken;
}
} else if (c == 'f' || c == 'F') {
count++;
c = line.charAt(count);
if (c == 'a') {
count++;
c = line.charAt(count);
}
if (c == 'l') {
count++;
c = line.charAt(count);
}
if (c == 's') {
count++;
c = line.charAt(count);
}
if (c == 'e') {
count++;
c = line.charAt(count);
}
if (c == ' ' || c == '!' || c == '!' || c == '^' || c == '(' || c == ')'){
token.add("false");
nextToken = "false";
// count++;
return nextToken;
}
} else if (c == ' ') {
count++;
} else {
count++;
}
}
return nextToken;
}
#Override
// provided, do not change
public void remove() {
// TODO Auto-generated method stub
throw new UnsupportedOperationException();
}
// provided
public static void main(String[] args) {
String line;
// you can play with other inputs on the command line
if (args.length > 0)
line = args[0];
// or do the standard test
else
line = " ! BAD (true ^ false) % truelybad";
System.out.println("line: [" + line + "]");
TokenIter tokIt = new TokenIter(line);
while (tokIt.hasNext())
System.out.println("next token: [" + tokIt.next() + "]");
}
}

Problem with your code comes only when last digit is not a token.
Reason - You are checking hasNext() which is true it goes inside your code.You are not setting nextToken for this case so it uses your lask token and display it.
I updated your code to always return a value and check if value return is from token list then display otherwise ignore it.
public class test implements Iterator<String> {
static List<String> tokenList = Arrays.asList( "true", "&", "!", "(", "false", "^", "true", ")");
ArrayList<String> token = new ArrayList<String>();
static int count = 0;
// input line to be tokenized
private String line;
// the next Token, null if no next Token
private String nextToken;
// implement
public test(String line) {
this.line = line;
}
#Override
// implement
public boolean hasNext() {
// System.out.println(count);
return count < line.length();
}
#Override
// implement
public String next() {
while (hasNext()) {
char c = line.charAt(count);
if (c == '!' || c == '!' || c == '^' || c == '(' || c == ')') {
token.add(Character.toString(c));
count++;
nextToken = Character.toString(c);
return nextToken;
} else if (c == 't' || c == 'T') {
count++;
c = line.charAt(count);
if (c == 'r') {
count++;
c = line.charAt(count);
}
if (c == 'u') {
count++;
c = line.charAt(count);
}
if (c == 'e') {
count++;
c = line.charAt(count);
}if (c == ' ' || c == '!' || c == '!' || c == '^' || c == '(' || c == ')'){
token.add("true");
nextToken = "true";
//count++;
return nextToken;
}
} else if (c == 'f' || c == 'F') {
count++;
c = line.charAt(count);
if (c == 'a') {
count++;
c = line.charAt(count);
}
if (c == 'l') {
count++;
c = line.charAt(count);
}
if (c == 's') {
count++;
c = line.charAt(count);
}
if (c == 'e') {
count++;
c = line.charAt(count);
}
if (c == ' ' || c == '!' || c == '!' || c == '^' || c == '(' || c == ')'){
token.add("false");
nextToken = "false";
// count++;
return nextToken;
}
} else if (c == ' ') {
count++;
nextToken = null;
} else {
count++;
nextToken = null;
}
}
return nextToken;
}
#Override
// provided, do not change
public void remove() {
// TODO Auto-generated method stub
throw new UnsupportedOperationException();
}
// provided
public static void main(String[] args) {
String line;
// you can play with other inputs on the command line
if (args.length > 0)
line = args[0];
// or do the standard test
else
line = " ! BAD (true ^ false) % truelybad ";
System.out.println("line: [" + line + "]");
test tokIt = new test(line);
while (tokIt.hasNext()) {
String s = tokIt.next();
if (s != null && tokenList.contains(s))
System.out.println("next token: [" + s + "]");
}
}
}

The underlying problem here is that your hasNext() method returns true not if there is another token in the String, but if it hasn't finished parsing the String yet.
So what happens is if you put in the String " ! ! true lotsofcrap ", then calling next() will return "!", then "!", then "true", then after that has been returned, there are no more tokens in the String, yet hasNext() still returns true.
What you might consider doing is having hasNext() parse through the string, but instead of returning the next String, return true only if it finds another token ahead of the current position. Keep in mind that in hasNext(), you do not want to directly increment count. Instead, make a local variable int something = count; at the beginning of hasNext() and use that. If you fix that, then the rest of your code SHOULD work just fine.

Custom Tokenizer, Iterator with quotes

Maybe some one can help?
How to modify this method next() that the next token can be: 'abc' text with the quotes.
Now if the text contains quote are throwed ExpressionException Unknown operator ''' at position...
#Override
public String next() {
StringBuilder token = new StringBuilder();
if (pos >= input.length()) {
return previousToken = null;
}
char ch = input.charAt(pos);
while (Character.isWhitespace(ch) && pos < input.length()) {
ch = input.charAt(++pos);
}
if (Character.isDigit(ch)) {
while ((Character.isDigit(ch) || ch == decimalSeparator)
&& (pos < input.length())) {
token.append(input.charAt(pos++));
ch = pos == input.length() ? 0 : input.charAt(pos);
}
} else if (ch == minusSign
&& Character.isDigit(peekNextChar())
&& ("(".equals(previousToken) || ",".equals(previousToken)
|| previousToken == null || operators
.containsKey(previousToken))) {
token.append(minusSign);
pos++;
token.append(next());
} else if (Character.isLetter(ch)) {
while ((Character.isLetter(ch) || Character.isDigit(ch) || (ch == '_')) && (pos < input.length())) {
token.append(input.charAt(pos++));
ch = pos == input.length() ? 0 : input.charAt(pos);
}
} else if (ch == '(' || ch == ')' || ch == ',') {
token.append(ch);
pos++;
//FIXME
else if (ch == '\''){
pos++;
String temp = "\'"+next()+"\'";
token.append(temp);
pos++;
}
//
} else {
while (!Character.isLetter(ch) && !Character.isDigit(ch)
&& !Character.isWhitespace(ch) && ch != '('
&& ch != ')' && ch != ',' && (pos < input.length())) {
token.append(input.charAt(pos));
pos++;
ch = pos == input.length() ? 0 : input.charAt(pos);
if (ch == minusSign) {
break;
}
}
if (!operators.containsKey(token.toString())) {
throw new ExpressionException("Unknown operator '" + token
+ "' at position " + (pos - token.length() + 1));
}
}
return previousToken = token.toString();
}
eval
public Object eval() {
Stack<Object> stack = new Stack<Object>();
for (String token : getRPN()) {
mylog.pl("Reverse polish notation TOKEN : " + token + " RPN size: " + getRPN().size() );
if (operators.containsKey(token)) {
Object v1 = stack.pop();
Object v2 = stack.pop();
stack.push(operators.get(token).eval(v2, v1));
} else if (variables.containsKey(token)) {
stack.push(variables.get(token).round(mc));
} else if (functions.containsKey(token.toUpperCase())) {
Function f = functions.get(token.toUpperCase());
ArrayList<Object> p = new ArrayList<Object>(f.getNumParams());
for (int i = 0; i < f.numParams; i++) {
p.add(0, stack.pop());
}
Object fResult = f.eval(p);
stack.push(fResult);
} else if (isDate(token)) {
Long date = null;
try {
date = SU.sdf.parse(token).getTime();
} catch (ParseException e) {/* IGNORE! */
}
stack.push(new BigDecimal(date, mc));
} else {
if (BusinessStrategy.PREFIX_X.equals(Character.toString(token.charAt(0)))) {
stack.push(token);
} else {
stack.push(new BigDecimal(token, mc));
}
}
}
return stack.pop();
}
Reverse notation
private List<String> getRPN() {
if (rpn == null) {
rpn = shuntingYard(this.expression);
}
return rpn;
}
Yard
private List<String> shuntingYard(String expression) {
List<String> outputQueue = new ArrayList<String>();
Stack<String> stack = new Stack<String>();
Tokenizer tokenizer = new Tokenizer(expression);
String lastFunction = null;
while (tokenizer.hasNext()) {
String token = tokenizer.next();
if (isNumber(token)) {
outputQueue.add(token);
} else if (variables.containsKey(token)) {
outputQueue.add(token);
} else if (functions.containsKey(token.toUpperCase())) {
stack.push(token);
lastFunction = token;
} else if (Character.isLetter(token.charAt(0))) {
if ("\'".equals(Character.toString(token.charAt(0)))){
outputQueue.add(token);
} else {
stack.push(token);
}
} else if (",".equals(token)) {
while (!stack.isEmpty() && !"(".equals(stack.peek())) {
outputQueue.add(stack.pop());
}
if (stack.isEmpty()) {
throw new ExpressionException("Parse error for function '"
+ lastFunction + "'");
}
} else if (operators.containsKey(token)) {
Operator o1 = operators.get(token);
String token2 = stack.isEmpty() ? null : stack.peek();
while (operators.containsKey(token2)
&& ((o1.isLeftAssoc() && o1.getPrecedence() <= operators
.get(token2).getPrecedence()) || (o1
.getPrecedence() < operators.get(token2)
.getPrecedence()))) {
outputQueue.add(stack.pop());
token2 = stack.isEmpty() ? null : stack.peek();
}
stack.push(token);
} else if ("(".equals(token)) {
stack.push(token);
} else if (")".equals(token)) {
while (!stack.isEmpty() && !"(".equals(stack.peek())) {
outputQueue.add(stack.pop());
}
if (stack.isEmpty()) {
throw new RuntimeException("Mismatched parentheses");
}
stack.pop();
if (!stack.isEmpty()
&& functions.containsKey(stack.peek().toUpperCase())) {
outputQueue.add(stack.pop());
}
}
}
while (!stack.isEmpty()) {
String element = stack.pop();
if ("(".equals(element) || ")".equals(element)) {
throw new RuntimeException("Mismatched parentheses");
}
if (!operators.containsKey(element)) {
throw new RuntimeException("Unknown operator or function: "
+ element);
}
outputQueue.add(element);
}
return outputQueue;
}
Error
*java.util.EmptyStackException
at java.util.Stack.peek(Unknown Source)
at java.util.Stack.pop(Unknown Source)
at com.business.Expression.eval(Expression.java:1033)*
It is in eval method Object v1 = stack.pop(); line.
Thanks !

In method next you have recursive calls in two places:
after seeing a minus sign
after recognizing an apostrope
The first situation will construct tokens where a minus is followed by a digit (i.e., an unsigend number follows) - OK. (Although, not having a sign but an unary minus operator deserves some consideration.)
The second scenario means trouble. After advancing past the initial apostrophe, another next-result is expected, as if string literals would only contain one number or one identifier or a single operator. Anyway, the next() executes, let's say it returns a number: then an apostroph is added to the token, but there's no effort to check whether there is a closing apostrophe nor to skip that.
else if (ch == '\''){
token.append( '\'' );
pos++;
while( pos < input.length() &&
(ch = input.charAt(pos++)) != '\'' ){
token.append( ch );
}
token.append( '\'' );
This doesn't permit an apostrophe to be a character within the string and it does not diagnose an unterminated string. But this can be added rather easily.