Python

These snippets do everything from creating singleton classes to merging dictionaries. Let’s enforce some Python best practices!

Add to Pieces

When downloading this collection to Pieces, developers will receive relevant tags for easy and fast search in Pieces, a description for context, and related links.

Remove falsy

Tags: python, remove falsy, list

Removes falsy values from a list by using the filter function.

def remove_falsy(unfiltered_list):
    return list(filter(bool, unfiltered_list)) 

Related links:

  1. https://devsheet.com/code-snippet/python-code-to-remove-falsy-valuesnone-false-etc/
  2. https://www.geeksforgeeks.org/remove-falsy-values-from-a-list-in-python/

Flatten

Tags: python, list, flatten, recursion

Flattens a nested list using recursion.

def flatten_list(nested_list):
    if not(bool(nested_list)):
        return nestedList
 
    if isinstance(nested_list[0], list):
        return flatten_list(*nested_list[:1]) + flatten_list(nested_list[1:])
 
    return nested_list[:1] + flatten_list(nested_list[1:]) 

Related links:

  1. https://www.tutorialspoint.com/python-program-to-flatten-a-nested-list-using-recursion
  2. https://www.geeksforgeeks.org/python-program-to-flatten-a-nested-list-using-recursion/

Check for duplicates

Tags: python, list, check duplicates

Checks whether a list has duplicate values by using set to grab only unique elements.

def check_for_duplicates(input):
    return len(input) != len(set(input))

Related links:

  1. https://www.w3schools.com/python/python_sets.asp
  2. https://docs.python.org/3/library/stdtypes.html#set-types-set-frozenset

Memory usage

Tags: python, memory usage, memory

Checks the total memory an object consumes.

import sys

a = 100;
print(sys.getsizeof(a)) 

Related links:

  1. https://stackoverflow.com/questions/33978/find-out-how-much-memory-is-being-used-by-an-object-in-pythonhttps://www.geeksforgeeks.org/find-out-how-much-memory-is-being-used-by-an-object-in-python/
  2. https://stackoverflow.com/questions/17574076/what-is-the-difference-between-len-and-sys-getsizeof-methods-in-python

Sort a list of dictionaries

Tags: python, dictionaries

Sorts dictionaries.

new_dict = {k: v for k, v in sorted(d.items(), key=lambda item: item[1])} 

Related links:

  1. https://pythonguides.com/python-dictionary-sort/
  2. https://favtutor.com/blogs/python-sort-dictionary

Merge any number of dictionaries

Tags: python, merge dictionary, dictionary

Merge multiple dictionaries into one.

def merge_dicts(*dicts):
  super_dict = {}
  for dict in dicts:
      for k, v in dict.items():
          super_dict[k] = v

 return super_dict

Related links:

  1. https://www.freecodecamp.org/news/merge-dictionaries-in-python/
  2. https://towardsdatascience.com/merge-dictionaries-in-python-d4e9ce137374

Check if a file exists

Tags: python, file

Checks if a file exists or not.

from os import path

def check_for_file(file_name):
	print("File exists: ", path.exists(file_name))

Related links:

  1. https://www.pythontutorial.net/python-basics/python-check-if-file-exists/
  2. https://flexiple.com/python/python-check-if-file-exists#section5

Merge Any Numbers of lists into lists of lists

Tags: python, file

Merge a series of lists into a list of lists. For example, [1, 2, 3] and [4, 5, 6] would merge into [[1, 2, 3], [4, 5, 6]].

def merge(*args, missing_val = None):
  max_length = max([len(lst) for lst in args])
  out_list = []

  for i in range(max_length):
    out_list.append([args[k][i] if i < len(args[k]) else missing_val for k in range(len(args))])

  return out_list 

Related links:

  1. https://builtin.com/data-science/python-code-snippets
  2. https://www.geeksforgeeks.org/args-kwargs-python/

Parse data from a CSV

Tags: python, csv

Parse CSV data and store it in a list of dictionaries.

import csv

def parse_csv_data(csv_path):
	csv_mapping_list = []

	with open(csv_path) as my_data:
	    csv_reader = csv.reader(my_data, delimiter=",")
	    line_count = 0
	    for line in csv_reader:
	        if line_count == 0:
	            header = line
	        else:
	            row_dict = {key: value for key, value in zip(header, line)}
	            csv_mapping_list.append(row_dict)
	        line_count += 1 

Related links:

  1. https://documentation.signavio.com/suite/en-us/Content/workflow-accelerator/developerguide/cookbook/parsing-csv-data.htm#:~:text=Comma Separated Values (CSV) is,data using the csv library.
  2. https://www.journaldev.com/30140/parse-csv-files-in-python

List Comprehension with Conditionals

Tags: python, list comprehension

Perform different operations in a list comprehension depending on a conditional.

l = [-1, 3, -4, 5, 6, -9]
l = [x if x >= 0 else 0 for x in l] 

Related links:

  1. https://stackoverflow.com/questions/4406389/if-else-in-a-list-comprehension
  2. https://www.w3schools.com/python/python_lists_comprehension.asp
  3. https://realpython.com/list-comprehension-python/

Write JSON data to file

Tags: python, json, json write

Write JSON data to a file.

import json

def write_json_to_file(data, filepath):
	with open(filename, "w") as f:
		json.dump(data, f, indent=4) 

Related links:

  1. https://stackoverflow.com/questions/12309269/how-do-i-write-json-data-to-a-file
  2. https://pythonexamples.org/python-write-json-to-file/

Read JSON data from a file

Tags: python, json, json read

Read JSON data from a file.

import json

def read_json_from_file(filepath):
	with open(filepath, "r") as f:
    data = json.load(f) 

Related links:

  1. https://www.geeksforgeeks.org/read-json-file-using-python/
  2. https://www.programiz.com/python-programming/json

Make an ABC Class

Tags: python, abc, abstract base class

Creates an abstract base class to test if objects adhere to given specifications.

from abc import ABCMeta, abstractmethod


class BaseClass(metaclass=ABCMeta):
    @abstractmethod
    def foo(self):
        pass

    @abstractmethod
    def bar(self):
        pass


class ConcreteClass(BaseClass):
    def foo(self):
        pass

    def bar(self):
        pass


instance = ConcreteClass() 

Related links:

  1. https://docs.python.org/3/library/abc.html
  2. https://www.geeksforgeeks.org/abstract-base-class-abc-in-python/

Thread Pool

Tags: python, thread pool, threads

Creates a thread pool.

import threading
import time
from queue import Queue


def f(n):
    time.sleep(n)


class Worker(threading.Thread):
    def __init__(self, queue):
        super(Worker, self).__init__()
        self.q = queue
        self.daemon = True
        self.start()

    def run(self):
        while 1:
            f, args, kwargs = self.q.get()
            try:
                f(*args, **kwargs)
            except Exception as e:
                print(e)
            self.q.task_done()


class ThreadPool(object):
    def __init__(self, thread_num=10):
        self.q = Queue(thread_num)
        for i in range(thread_num):
            Worker(self.q)

    def add_task(self, f, *args, **kwargs):
        self.q.put((f, args, kwargs))

    def wait_complete(self):
        self.q.join()


if __name__ == '__main__':
    start = time.time()
    pool = ThreadPool(5)
    for i in range(10):
        pool.add_task(f, 3)
    pool.wait_complete()
    end = time.time() 

Related links:

  1. https://www.tutorialspoint.com/concurrency_in_python/concurrency_in_python_pool_of_threads.htm
  2. https://superfastpython.com/threadpoolexecutor-in-python/

Set up a virtual environment

Tags: python, virtual environment

Create a virtual environment in Python to manage dependencies.

python -m venv projectnamevenv 

Related links:

  1. https://docs.python.org/3/library/venv.html
  2. https://realpython.com/python-virtual-environments-a-primer/
  3. https://www.freecodecamp.org/news/how-to-setup-virtual-environments-in-python/

Singleton decorator

Tags: python, virtual environment

Decorator to create a singleton class.

def singleton(myClass):
	instances = {}
	def getInstance(*args, **kwargs):
		if myClass not in instances:
			instances[myClass] = myClass(*args, **kwargs)
		return instances[myClass]
	return getInstance

@singleton
class TestClass(object):
	pass 

Related links:

  1. https://www.geeksforgeeks.org/singleton-pattern-in-python-a-complete-guide/#:~:text=A Singleton pattern in python,of access for a resource.
  2. https://www.tutorialspoint.com/python_design_patterns/python_design_patterns_singleton.htm

Stream

Tags: python, stream

Creates a stream.

def processor(reader, converter, writer):
    while True:
        data = reader.read()
        if not data:
            break
        data = converter(data)
        writer.write(data)


class Processor:
    def __init__(self, reader, writer):
        self.reader = reader
        self.writer = writer

    def process(self):
        while True:
            data = self.reader.readline()
            if not data:
                break
            data = self.converter(data)
            self.writer.write(data)

    def converter(data):
        assert False, 'converter must be defined' 

Related links:

  1. https://docs.python.org/3/library/asyncio-stream.html
  2. https://livebook.manning.com/book/concurrency-in-python-with-asyncio/chapter-8

Logging

Tags: python, logging

Set logging for debug level along with file name and output to a sample.log file.

import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s:%(name)s:%(message)s')

file_handler = logging.FileHandler('sample.log')
file_handler.setLevel(logging.ERROR)
file_handler.setFormatter(formatter)

stream_handler = logging.StreamHandler()
stream_handler.setFormatter(formatter)

logger.addHandler(file_handler)
logger.addHandler(stream_handler) 

Related links:

  1. https://docs.python.org/3/howto/logging.html
  2. https://realpython.com/python-logging/