How can I improve this Python / Xonsh script (converted from bash)?

Question

I'm starting to learn xonsh and Python. I want to use Python mainly for shell scripting and I want to replace many existing small to medium bash scripts.

I started with the bash script given below (with some comments added). I manually converted it into Python (shown further below). Please point out where my Python can be improved or made more concise.

#!/bin/bash while IFS= read -r line; do echo $line | grep '^Path=' >/dev/null 2>&1 if [[ $? -eq 0 ]]; then \ path=~/.mozilla/firefox/$(echo $line | sed 's/^Path=//')/user.js; #open the user.js file for each profile if [[ -f ${path} ]]; then \ echo "Processing: path: ${path}"; grep 'browser.ctrlTab.sortByRecentlyUsed' ${path} >/dev/null 2>&1 #check if this property already exists if [ $? -ne 0 ]; then \ echo 'user_pref("browser.ctrlTab.sortByRecentlyUsed", true);' >> ${path}; #did not exist, so append it to end of file else sed -i 's/browser.ctrlTab.sortByRecentlyUsed",.*false/browser.ctrlTab.sortByRecentlyUsed", true/' ${path} #did exist, so change it to desired value if needed fi echo "updated ctrlTab.sortByRecentlyUsed for $path" grep 'browser.tabs.loadBookmarksInTabs' ${path} >/dev/null 2>&1 if [ $? -ne 0 ]; then \ echo 'user_pref("browser.tabs.loadBookmarksInTabs", false);' >> ${path}; else sed -i 's/browser.tabs.loadBookmarksInTabs",.*true/browser.tabs.loadBookmarksInTabs", false/' ${path} fi echo "updated loadBookmarksInTabs for $path" fi fi done < ~/.mozilla/firefox/profiles.ini #read this file to find the path to all profiles 

I have a working Python solution now, but it seems a lot more complicated that I expected. My goal is to be able to write short & quick solutions for situations like this one. At this time I don't expect to be writing large Python programs. I do hope to use xonsh extensively once I learn enough. I can obviously get rid of the print statements I included for my learning. Other than that, how can my solution shown below be improved or shortened?

In particular, I had intended to use pysed and Python's fileinput but I wasn't able to achieve a working solution with those.

import sys from pathlib import Path import re trgt_file='user.js' myuser='mountainx' base_path = Path('/home/' myuser '/.mozilla/firefox') if not base_path.is_dir(): print("ERROR:", base_path, "does not exist.") # exit() else: print("OK: processing", base_path) ini_path = Path(base_path / 'profiles.ini') if not ini_path.is_file(): print("ERROR:", ini_path, "cannot be opened.") # exit() else: print("OK: opening", ini_path) match_ctrltab = False match_bookmark_found = False pro_path_re = re.compile(r'^Path=(\w+)$') ctrltab_sort_regex = re.compile(r'^/*?user_pref\("browser\.ctrlTab\.sortByRecentlyUsed.*$', re.M) ctrltab_sort_repls = 'user_pref("browser.ctrlTab.sortByRecentlyUsed", true);' bookmark_open_regex = re.compile(r'^/*?user_pref\("browser\.tabs\.loadBookmarksInTabs.*$', re.M) bookmark_open_repls = 'user_pref("browser.tabs.loadBookmarksInTabs", false);' with open(ini_path, "r") as profiles_ini: for any_line in profiles_ini: m_path = pro_path_re.match(any_line) if not m_path is None: p_path = m_path.group(1) print("p_path:", p_path) profile_path = Path(base_path / p_path) print("profile_path:", profile_path) if profile_path.is_dir(): print("The profile path is", profile_path) user_js_path = Path(profile_path / trgt_file) if Path(user_js_path).is_file(): print("OK: valid file:", user_js_path) with open(user_js_path, 'r+') as file_userjs: #read the file contents file_contents = file_userjs.read() match_ctrltab = ctrltab_sort_regex.search(file_contents) if match_ctrltab is None: file_contents = file_contents + '\n' + ctrltab_sort_repls print('No match: added line to end of file contents:', ctrltab_sort_repls) else: file_contents = ctrltab_sort_regex.sub(ctrltab_sort_repls, file_contents) print('Match found. Replaced line in file for ctrltab_sort') match_bookmark = bookmark_open_regex.search(file_contents) if match_bookmark is None: file_contents = file_contents + '\n' + bookmark_open_repls print('No match: added line to end of file contents:', bookmark_open_repls) else: file_contents = bookmark_open_regex.sub(bookmark_open_repls, file_contents) print('Match found. Replaced line for bookmark_open_regex') file_userjs.seek(0) file_userjs.truncate() file_userjs.write(file_contents) else: print("SKIPPED: invalid file:", user_js_path) 

Here is some date I am testing it against. The general solution should work against any Firefox user.js or prefs.js file.

file_contents = """ user_pref("media.gmp-gmpopenh264.abi", "x86_64-gcc3"); user_pref("media.gmp-gmpopenh264.autoupdate", false); user_pref("media.gmp-gmpopenh264.enabled", false); user_pref("media.gmp-gmpopenh264.lastUpdate", 1571947201); user_pref("media.gmp-gmpopenh264.version", "1.8.1"); user_pref("browser.ctrlTab.sortByRecentlyUsed",false); //user_pref("browser.ctrlTab.sortByRecentlyUsed", true);""" 

Answer 1

Here's the best I have come up with on my own so far.

I wrote two functions and saved them in a directory named modtest (and saved the file as ffutils.py).
touch modtest/__init__.py
See https://stackoverflow.com/a/33770042 for more on that.

import sys from pathlib import Path import re def find_profile_dirs(user=None, debug=False): profiles = [] if user is None: base_path = Path('~/.mozilla/firefox').expanduser() else: base_path = Path('/home') / user / '.mozilla/firefox' if not base_path.is_dir(): print("ERROR:", base_path, "does not exist.") elif debug: print("OK: processing", base_path) ini_path = Path(base_path / 'profiles.ini') if not ini_path.is_file(): print("ERROR:", ini_path, "cannot be opened.") elif debug: print("OK: opening", ini_path) pro_path_re = re.compile(r'^Path=(\w+)$') with open(ini_path, "r") as profiles_ini: for any_line in profiles_ini: m_path = pro_path_re.match(any_line) if not m_path is None: p_path = m_path.group(1) if debug: print("p_path:", p_path) profile_path = Path(base_path / p_path) if debug: print("profile_path:", profile_path) if profile_path.is_dir(): profiles.append(profile_path) print("The profile path is", profile_path) return profiles # profiles_paths is a list of Paths of all Firefox profiles to process def process_userjs(searchExp, replaceExp, profiles_paths, debug=False): match_result = None search_regex = re.compile(searchExp, re.M) trgt_file='user.js' for profile_path in profiles_paths: user_js_path = Path(profile_path / trgt_file) if Path(user_js_path).is_file(): if debug: print("OK: valid file:", user_js_path) with open(user_js_path, 'r+') as file_userjs: file_contents = file_userjs.read() match_result = search_regex.search(file_contents) if match_result is None: file_contents = file_contents + '\n' + replaceExp if debug: print('No match: added new line to end of file contents:', replaceExp) else: file_contents = search_regex.sub(replaceExp, file_contents) if debug: print('Match found. Modified existing line in file.') file_userjs.seek(0) file_userjs.truncate() file_userjs.write(file_contents) elif debug: print("SKIPPED: invalid file:", user_js_path) 

Then I used the functions like this:

import sys sys.path = ['/path/to/parent/of/modtest'] + sys.path # if needed from modtest import ffutils my_profiles = ffutils.find_profile_dirs() srchexp = r'^/*?user_pref\("browser\.ctrlTab\.sortByRecentlyUsed.*$' replexp = 'user_pref("browser.ctrlTab.sortByRecentlyUsed", true);' ffutils.process_userjs(srchexp, replexp, my_profiles, True) 

Of course, I can repeat it with other search & replace strings.

I'm sure there are still many ways to improve this. I'll accept other answers, but I'm putting my own answer here in case there are no others offered.

There are a few assumptions in my code. A big one is that it assumes all Firefox profile paths are relative. If that isn't a valid assumption, a little more processing on profiles.ini is needed.