Parse file and make output in custom List [closed]

Question

I have a file output like this and want to do parsing and put the output into a List. The biggest problem here is the name of a wake-lock which appears after ID and runtime, in some cases, it contains spaces, ":" or () which made me hard to parse. Some cases also don't have runtime neither counts, instead all of that there is just word realtime. I have put my approach below, maybe it needs just a little improvement which I cannot see, or it a whole miss.

1000 ActivityManager-Sleep: 4s 493ms (0 times) max=4991 actual=4991 realtime u0a149 AudioIn: 1s 274ms (1 times) max=1274 realtime u0a138 *job*/com.android.vending/com.google.android.finsky.scheduler.process.mainimpl.PhoneskyJobServiceMain: 257ms (1 times) max=547 actual=547 realtime 1041 AudioIn: 133ms (3 times) max=126 realtime u0a136 GCoreFlp: 81ms (1 times) max=153 actual=153 realtime 1001 *telephony-radio*: 76ms (3 times) max=155 actual=157 realtime u0a188 Doze: 39ms (1 times) max=124 actual=124 realtime u0a188 Scrims: 36ms (1 times) max=115 actual=115 realtime u0a257 *job*/com.liverpool.echo/com.urbanairship.job.AndroidJobService: 32ms (1 times) max=96 actual=96 realtime 1000 startDream: 25ms (1 times) max=59 actual=59 realtime 1000 GCoreFlp: 23ms (1 times) max=84 actual=84 realtime u0a188 show keyguard: 22ms (0 times) max=48 actual=48 realtime u0a136 GCM_READ: 17ms (2 times) max=12 realtime 1000 AnyMotionDetector: 17ms (1 times) max=24 actual=24 realtime u0a145 *job*/com.google.android.apps.turbo/.nudges.broadcasts.BatteryHistoryLoggerJobService: 11ms (1 times) max=47 actual=47 realtime 1000 deviceidle_going_idle: 10ms (1 times) max=32 actual=32 realtime u0a136 NlpWakeLock: 9ms (6 times) max=11 actual=36 realtime u0a136 CMWakeLock: 8ms (1 times) max=13 actual=13 realtime 1002 bluetooth_timer: 8ms (4 times) max=8 actual=13 realtime u0a136 UlrDispSvcFastWL: 6ms (3 times) max=19 actual=23 realtime 1000 *alarm*: 6ms (1 times) max=6 realtime u0a136 *alarm*: 6ms (1 times) max=9 actual=9 realtime 1000 NlpWakeLock: 4ms (3 times) max=8 actual=13 realtime u0a136 GCM_HB_ALARM: 3ms (1 times) max=6 actual=6 realtime u0a166 GCoreFlp: 3ms (2 times) max=3 actual=6 realtime u0a145 *job*/com.google.android.apps.turbo/.deadline.library.DeadlineUpdateJobService: 3ms (1 times) max=11 actual=11 realtime u0a147 NlpWakeLock: 2ms (3 times) max=4 actual=8 realtime 1000 GnssLocationProvider: 2ms (4 times) max=4 actual=7 realtime 1000 WifiSuspend: 2ms (1 times) max=7 actual=7 realtime u0a136 *gms_scheduler*:internal: 1ms (1 times) max=5 actual=5 realtime u0a136 Wakeful StateMachine: GeofencerStateMachine: 1ms (1 times) max=2 actual=2 realtime 1027 NfcService:mRoutingWakeLock realtime 

I need an Array output for each line with ID, name, runTime, timesTriggered, max, actual

I tried to do this with split and substring, but I don't like my approach, also it won't work at some cases.

 for (int k = 0; k < wakelocksToParse.size(); k++) { String wakelockStat = wakelocksToParse.get(k); utils.writeFile(Data.PARTIAL_WAKELOCKS_NEW, wakelockStat, true); if (wakelockStat.lastIndexOf(":") != -1) { String UID; String wakelockName; String wakelockTime; String wakelockCount; long timeInMs = 0; UID = wakelockStat.substring(0, wakelockStat.indexOf(" ")); wakelockName = wakelockStat.substring(wakelockStat.indexOf(" "), wakelockStat.lastIndexOf(":")).trim(); if (wakelockName.length() == 0) wakelockName = "unknown"; wakelockTime = wakelockStat.substring(wakelockStat.lastIndexOf(":") + 2, wakelockStat.indexOf("(") - 1); String[] time = wakelockTime.replaceAll("[^0-9 ]", "").split(" "); if (time.length == 5) { int days = utils.parseIntWithDefault(time[0], 0); int hours = utils.parseIntWithDefault(time[1], 0); int minutes = utils.parseIntWithDefault(time[2], 0); int seconds = utils.parseIntWithDefault(time[3], 0); int ms = utils.parseIntWithDefault(time[4], 0); timeInMs = (days * 86400000) + (hours * 3600000) + (minutes * 60000) + (seconds * 1000) + ms; } else if (time.length == 4) { int hours = utils.parseIntWithDefault(time[0], 0); int minutes = utils.parseIntWithDefault(time[1], 0); int seconds = utils.parseIntWithDefault(time[2], 0); int ms = utils.parseIntWithDefault(time[3], 0); timeInMs = (hours * 3600000) + (minutes * 60000) + (seconds * 1000) + ms; } else if (time.length == 3) { int minutes = utils.parseIntWithDefault(time[0], 0); int seconds = utils.parseIntWithDefault(time[1], 0); int ms = utils.parseIntWithDefault(time[2], 0); timeInMs = (minutes * 60000) + (seconds * 1000) + ms; } else if (time.length == 2) { int seconds = utils.parseIntWithDefault(time[0], 0); int ms = utils.parseIntWithDefault(time[1], 0); timeInMs = (seconds * 1000) + ms; } else if (time.length == 1) { timeInMs = utils.parseIntWithDefault(time[0], 0); } wakelockCount = wakelockStat.substring(wakelockStat.lastIndexOf(':') + 1).trim(); wakelockCount = wakelockCount.substring(wakelockCount.indexOf("(") + 1); wakelockCount = wakelockCount.substring(0, wakelockCount.indexOf(" ")).trim(); Log.d("aaaa", UID + " " + wakelockName + " " + timeInMs + " " + wakelockCount); if (timeInMs >= 1000) data.add(new WakelocksData(UID, wakelockName, timeInMs, wakelockCount)); utils.writeFile(Data.PARTIAL_WAKELOCKS_OLD, UID + " " + wakelockName + " " + timeInMs + " " + wakelockCount, true); } } ```

Answer 1

This was an interesting problem. I suspect the purpose of this assignment was to explore different String methods.

The first thing I did was create a LogMessage class to hold the separated contents of one log message. The runTime value is in milliseconds.

Here are the test results from one of my many, many test runs.

LogMessage [runTime=4493, timesTriggered=0, maxTimes=4991, actualTimes=4991, logID=1000, logName=ActivityManager-Sleep: ] LogMessage [runTime=1274, timesTriggered=1, maxTimes=1274, actualTimes=0, logID=u0a149, logName=AudioIn: ] LogMessage [runTime=257, timesTriggered=1, maxTimes=547, actualTimes=547, logID=u0a138, logName=*job*/com.android.vending/com.google.android.finsky.scheduler.process.mainimpl.PhoneskyJobServiceMain: ] LogMessage [runTime=133, timesTriggered=3, maxTimes=126, actualTimes=0, logID=1041, logName=AudioIn: ] LogMessage [runTime=81, timesTriggered=1, maxTimes=153, actualTimes=153, logID=u0a136, logName=GCoreFlp: ] LogMessage [runTime=76, timesTriggered=3, maxTimes=155, actualTimes=157, logID=1001, logName=*telephony-radio*: ] LogMessage [runTime=39, timesTriggered=1, maxTimes=124, actualTimes=124, logID=u0a188, logName=Doze: ] LogMessage [runTime=36, timesTriggered=1, maxTimes=115, actualTimes=115, logID=u0a188, logName=Scrims: ] LogMessage [runTime=32, timesTriggered=1, maxTimes=96, actualTimes=96, logID=u0a257, logName=*job*/com.liverpool.echo/com.urbanairship.job.AndroidJobService: ] LogMessage [runTime=25, timesTriggered=1, maxTimes=59, actualTimes=59, logID=1000, logName=startDream: ] LogMessage [runTime=23, timesTriggered=1, maxTimes=84, actualTimes=84, logID=1000, logName=GCoreFlp: ] LogMessage [runTime=22, timesTriggered=0, maxTimes=48, actualTimes=48, logID=u0a188, logName=show keyguard: ] LogMessage [runTime=17, timesTriggered=2, maxTimes=12, actualTimes=0, logID=u0a136, logName=GCM_READ: ] LogMessage [runTime=17, timesTriggered=1, maxTimes=24, actualTimes=24, logID=1000, logName=AnyMotionDetector: ] LogMessage [runTime=11, timesTriggered=1, maxTimes=47, actualTimes=47, logID=u0a145, logName=*job*/com.google.android.apps.turbo/.nudges.broadcasts.BatteryHistoryLoggerJobService: ] LogMessage [runTime=10, timesTriggered=1, maxTimes=32, actualTimes=32, logID=1000, logName=deviceidle_going_idle: ] LogMessage [runTime=9, timesTriggered=6, maxTimes=11, actualTimes=36, logID=u0a136, logName=NlpWakeLock: ] LogMessage [runTime=8, timesTriggered=1, maxTimes=13, actualTimes=13, logID=u0a136, logName=CMWakeLock: ] LogMessage [runTime=8, timesTriggered=4, maxTimes=8, actualTimes=13, logID=1002, logName=bluetooth_timer: ] LogMessage [runTime=6, timesTriggered=3, maxTimes=19, actualTimes=23, logID=u0a136, logName=UlrDispSvcFastWL: ] LogMessage [runTime=6, timesTriggered=1, maxTimes=6, actualTimes=0, logID=1000, logName=*alarm*: ] LogMessage [runTime=6, timesTriggered=1, maxTimes=9, actualTimes=9, logID=u0a136, logName=*alarm*: ] LogMessage [runTime=4, timesTriggered=3, maxTimes=8, actualTimes=13, logID=1000, logName=NlpWakeLock: ] LogMessage [runTime=3, timesTriggered=1, maxTimes=6, actualTimes=6, logID=u0a136, logName=GCM_HB_ALARM: ] LogMessage [runTime=3, timesTriggered=2, maxTimes=3, actualTimes=6, logID=u0a166, logName=GCoreFlp: ] LogMessage [runTime=3, timesTriggered=1, maxTimes=11, actualTimes=11, logID=u0a145, logName=*job*/com.google.android.apps.turbo/.deadline.library.DeadlineUpdateJobService: ] LogMessage [runTime=2, timesTriggered=3, maxTimes=4, actualTimes=8, logID=u0a147, logName=NlpWakeLock: ] LogMessage [runTime=2, timesTriggered=4, maxTimes=4, actualTimes=7, logID=1000, logName=GnssLocationProvider: ] LogMessage [runTime=2, timesTriggered=1, maxTimes=7, actualTimes=7, logID=1000, logName=WifiSuspend: ] LogMessage [runTime=1, timesTriggered=1, maxTimes=5, actualTimes=5, logID=u0a136, logName=*gms_scheduler*:internal: ] LogMessage [runTime=1, timesTriggered=1, maxTimes=2, actualTimes=2, logID=u0a136, logName=Wakeful StateMachine: GeofencerStateMachine: ] LogMessage [runTime=0, timesTriggered=0, maxTimes=0, actualTimes=0, logID=1027, logName=NfcService:mRoutingWakeLock ] 

The code turned out to be rather involved. Here's a diagram of the internal method call tree.

main processLogFile parseLogLine parseLastPart append getTimesTriggered valueOf getRunTime valueOf getValue valueOf 

Here's the complete runnable code. I put your input file in my resources folder. You can delete the lines where I get the complete path to the file.

import java.io.BufferedReader; import java.io.File; import java.io.FileNotFoundException; import java.io.FileReader; import java.io.IOException; import java.util.ArrayList; import java.util.List; public class ProcessLogText { public static void main(String[] args) { ProcessLogText plt = new ProcessLogText(); List<LogMessage> logMessages = plt.processLogFile("log.txt"); for (LogMessage logMessage : logMessages) { System.out.println(logMessage); } } public List<LogMessage> processLogFile(String fileName) { List<LogMessage> logMessages = new ArrayList<>(); ClassLoader classLoader = getClass().getClassLoader(); File file = new File(classLoader.getResource(fileName).getFile()); try { BufferedReader reader = new BufferedReader(new FileReader(file)); String line = reader.readLine(); while (line != null) { logMessages.add(parseLogLine(line)); line = reader.readLine(); } reader.close(); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } return logMessages; } private LogMessage parseLogLine(String line) { int endIndex = line.lastIndexOf(": "); String firstPart, lastPart; if (endIndex >= 0) { firstPart = line.substring(0, endIndex + 2); lastPart = line.substring(endIndex + 2); } else { endIndex = line.lastIndexOf("realtime"); firstPart = line.substring(0, endIndex); lastPart = line.substring(endIndex); } endIndex = firstPart.indexOf(" "); String logID = firstPart.substring(0, endIndex); String logName = firstPart.substring(endIndex + 1); List<String> parts = parseLastPart(lastPart); int timesTriggered = getTimesTriggered(parts); int runTime = getRunTime(parts); int maxTimes = getValue(parts, "max"); int actualTimes = getValue(parts, "actual"); return new LogMessage(logID, logName, runTime, timesTriggered, maxTimes, actualTimes); } private List<String> parseLastPart(String lastPart) { List<String> parts = new ArrayList<>(); StringBuilder builder = new StringBuilder(); boolean insideParenthesis = false; for (int i = 0; i < lastPart.length(); i++) { char c = lastPart.charAt(i); if (c == '(') { insideParenthesis = true; continue; } if (c == ')') { insideParenthesis = false; continue; } if (c == ' ') { if (insideParenthesis) { builder.append(c); } else { append(parts, builder); } } else { builder.append(c); } } append(parts, builder); return parts; } private void append(List<String> parts, StringBuilder builder) { if (builder.length() > 0) { parts.add(builder.toString()); builder.delete(0, builder.length()); } } private int getTimesTriggered(List<String> parts) { int timesTriggered = 0; for (int i = parts.size() - 1; i >= 0; i--) { String s = parts.get(i); int endIndex = s.indexOf(" times"); if (endIndex >= 0) { int number = valueOf(s.substring(0, endIndex)); if (number >= 0) { timesTriggered += number; } parts.remove(i); break; } } return timesTriggered; } private int getRunTime(List<String> parts) { int runTime = 0; for (String s : parts) { int endIndex = s.indexOf("ms"); if (endIndex >= 0) { int number = valueOf(s.substring(0, endIndex)); if (number >= 0) { runTime += number; } } else { endIndex = s.indexOf("s"); if (endIndex >= 0) { int number = valueOf(s.substring(0, endIndex)); if (number >= 0) { runTime += number * 1000; } } } } return runTime; } private int getValue(List<String> parts, String key) { key = key + "="; for (String s : parts) { if (s.indexOf(key) == 0) { return Math.max(0, valueOf(s.substring(key.length()))); } } return 0; } private int valueOf(String number) { try { return Integer.valueOf(number); } catch (NumberFormatException e) { return -1; } } public class LogMessage { private final int runTime; private final int timesTriggered; private final int maxTimes; private final int actualTimes; private final String logID; private final String logName; public LogMessage(String logID, String logName, int runTime, int timesTriggered, int maxTimes, int actualTimes) { this.logID = logID; this.logName = logName; this.runTime = runTime; this.timesTriggered = timesTriggered; this.maxTimes = maxTimes; this.actualTimes = actualTimes; } public String getLogID() { return logID; } public String getLogName() { return logName; } public int getRunTime() { return runTime; } public int getTimesTriggered() { return timesTriggered; } public int getMaxTimes() { return maxTimes; } public int getActualTimes() { return actualTimes; } @Override public String toString() { StringBuilder builder = new StringBuilder(); builder.append("LogMessage [runTime="); builder.append(runTime); builder.append(", timesTriggered="); builder.append(timesTriggered); builder.append(", maxTimes="); builder.append(maxTimes); builder.append(", actualTimes="); builder.append(actualTimes); builder.append(", logID="); builder.append(logID); builder.append(", logName="); builder.append(logName); builder.append("]"); return builder.toString(); } } } 

So, here's what I did to parse the log message file.

1. I split the message into two parts. The first part contains the message-id and message-name. The second part contains the remainder of the log message. I was able to split on either ": " or "realtime". I didn't know whether the final colon was part of the message-name or not, so I left it in.

2. I split the first part on the first space and got the message-id and message-name.

3. I parsed the last part into pieces based on a space outside of the parenthesis.

4. I got the times triggered next. I did this first so I could delete the part when I finished processing it. Deleting the times triggered part made calculating the run time much easier because I could check for "ms" and "s" without getting a false "s" on "times".

5. I calculated the run time from the parts.

6. I got the max value from the parts.

7. I got the actual value from the parts.

I have no idea how you might be able to do this with one or more regular expressions.

By doing it this way, I was able to test each step along the way and fix any problems I found before I proceeded to the next step. Like the last log message not having a colon.

Answer 2

I want to post my final answer based on @Gilbert Le Blanc code, I have changed several things, but base is the same.

 private void parseWakelockStat(String wakelockStat) { int endIndex = wakelockStat.lastIndexOf(": "); String firstPart, lastPart; if (endIndex >= 0) { firstPart = wakelockStat.substring(0, endIndex + 2); lastPart = wakelockStat.substring(endIndex + 2); } else { endIndex = wakelockStat.lastIndexOf("realtime"); firstPart = wakelockStat.substring(0, endIndex); lastPart = wakelockStat.substring(endIndex); } endIndex = firstPart.indexOf(" "); List<String> parts = parseLastPart(lastPart); // UID of wakelocks (if exists, only partial wakelocks have UID) String UID = firstPart.substring(0, endIndex); // Wakelock name String wakelockName = firstPart.substring(endIndex + 1); // Runtime of wakelock int runTime = getRunTime(parts); // Wakelock trigger count int wakelockTriggerCount = getWakelockTriggerCount(parts); // Get max and actual wakelock count int maxTimes = getValue(parts, "max"); int actualTimes = getValue(parts, "actual"); wakelock = new Wakelock(UID, wakelockName, runTime, wakelockTriggerCount, maxTimes, actualTimes); } private List<String> parseLastPart(String lastPart) { List<String> parts = new ArrayList<>(); StringBuilder builder = new StringBuilder(); boolean insideParenthesis = false; for (int i = 0; i < lastPart.length(); i++) { char c = lastPart.charAt(i); if (c == '(') { insideParenthesis = true; continue; } if (c == ')') { insideParenthesis = false; continue; } if (c == ' ') { if (insideParenthesis) { builder.append(c); } else { append(parts, builder); } } else { builder.append(c); } } append(parts, builder); return parts; } private void append(List<String> parts, StringBuilder builder) { if (builder.length() > 0) { parts.add(builder.toString()); builder.delete(0, builder.length()); } } private int getWakelockTriggerCount(List<String> parts) { int timesTriggered = 0; for (int i = parts.size() - 1; i >= 0; i--) { String s = parts.get(i); int endIndex = s.indexOf(" times"); if (endIndex >= 0) { int number = utils.parseIntWithDefault(s.substring(0, endIndex), 0); if (number >= 0) { timesTriggered += number; } parts.remove(i); break; } } return timesTriggered; } private int getRunTime(List<String> parts) { int runTime = 0; for (String part : parts) { if (part.endsWith("d")) { int time = utils.parseIntWithDefault(part.replaceAll("[^0-9 ]", ""), 0); if (time >= 0) runTime += time * 86400000; } else if (part.endsWith("h")) { int time = utils.parseIntWithDefault(part.replaceAll("[^0-9 ]", ""), 0); if (time >= 0) runTime += time * 3600000; } else if (part.endsWith("m")) { int time = utils.parseIntWithDefault(part.replaceAll("[^0-9 ]", ""), 0); if (time >= 0) runTime += time * 60000; } else if (part.endsWith("s") && !part.endsWith("ms")) { int time = utils.parseIntWithDefault(part.replaceAll("[^0-9 ]", ""), 0); if (time >= 0) runTime += time * 1000; } else if (part.endsWith("ms")) { int time = utils.parseIntWithDefault(part.replaceAll("[^0-9 ]", ""), 0); if (time >= 0) runTime += time; break; } } return runTime; } private int getValue(List<String> parts, String key) { key = key + "="; for (String s : parts) { if (s.indexOf(key) == 0) { return Math.max(0, utils.parseIntWithDefault(s.substring(key.length()), 0)); } } return 0; } public static class Wakelock { private final String UID; private final String wakelockName; private final int runTime; private final int wakelockTriggerCount; private final int maxTimes; private final int actualTimes; public Wakelock(String UID, String wakelockName, int runTime, int wakelockTriggerCount, int maxTimes, int actualTimes) { this.UID = UID; this.wakelockName = wakelockName; this.runTime = runTime; this.wakelockTriggerCount = wakelockTriggerCount; this.maxTimes = maxTimes; this.actualTimes = actualTimes; } private String getUID() { return UID; } private String getWakelockName() { String name; if (wakelockName.trim().length() == 0) name = UID.trim(); else name = wakelockName.trim(); // Remove : from the end of the name if exists return name.endsWith(":") ? name.substring(0, name.length() - 1) : name; } private int getRunTime() { return runTime; } private int getWakelockTriggerCount() { return wakelockTriggerCount; } private int getMaxTimes() { return maxTimes; } private int getActualTimes() { return actualTimes; } } 

• The first thing I've done is removing ":" from the end of a wakelock name.
• Managed getRunTime(), I have to parse also days, minutes, hours insted of just seconds and ms, my example hadn't it. Also applied break the loop when part ends with ms, to avoid checking next parts. They are not needed.

And this is how I call whole function

for (int k = 0; k < wakelocksToParse.size(); k++) { String wakelockStat = wakelocksToParse.get(k); utils.writeFile(Data.KERNEL_WAKELOCKS_OLD, wakelockStat, true); parseWakelockStat(wakelockStat); utils.writeFile(Data.KERNEL_WAKELOCKS_NEW, wakelock.getUID() + " " + wakelock.getWakelockName() + " " + wakelock.getRunTime() + " " + wakelock.getWakelockTriggerCount() + " " + wakelock.getMaxTimes() + " " + wakelock.getActualTimes(), true); data.add(new WakelocksData(wakelock.getUID(), wakelock.getWakelockName(), wakelock.getRunTime(), wakelock.getWakelockTriggerCount())); }