Speed up make/cmake build with multiprocessing

Build time can be reduced by using multiple cores of your processor. Parallelization for make: make -j8 Parallelization for cmake: cmake --build <bindir> -- -j 8 Sometimes you cannot pass parameters directly to make. For example you are installing a python module using python setup.py install Setup script doesn’t accept parameters. Then you could pass them through environment variables: export MAKEFLAGS="-j 8" see also intro to CMake video, russian.

July 29, 2017 · SergeM

Add months to datetime in python

1 2 3 4 5 import datetime from dateutil.relativedelta import relativedelta your_datetime = datetime.datetime.now() your_datetime + relativedelta(months=1) # adds one month That function clips the overflowing day of the months: 1 2 3 4 5 6 7 8 9 10 >>> datetime(2021, 1, 28) + relativedelta(months=1) datetime.datetime(2021, 2, 28, 0, 0) >>> datetime(2021, 1, 29) + relativedelta(months=1) datetime.datetime(2021, 2, 28, 0, 0) >>> datetime(2021, 1, 30) + relativedelta(months=1) datetime.datetime(2021, 2, 28, 0, 0) >>> datetime(2021, 1, 31) + relativedelta(months=1) datetime....

July 20, 2017 · SergeM

Module import in python

Import module by path/name 1 2 import importlib module = importlib.import_module(module_path) Find class implementing certain interface in a module 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 def load_class_from_module(module_path: str, target_class: type) -> type: module = importlib.import_module(module_path) list_classes = get_all_classes_implementing_interface(module, target_class) if not list_classes: raise RuntimeError("Unable to find implemented interface {} in {}".format(target_class, module_path)) if len(list_classes) > 1: raise RuntimeError("More then 1 implementation of {} was found in {}"....

July 9, 2017 · SergeM

Argparse recipies

How to include default values in ‘–help’ 1 2 3 parser = argparse.ArgumentParser( # ... other options ... formatter_class=argparse.ArgumentDefaultsHelpFormatter) Output looks like this: 1 2 3 --scan-time [SCAN_TIME] Wait SCAN-TIME seconds between status checks. (default: 5) Another tip: add ‘%(default)’ to the help parameter to control what is displayed. 1 2 parser.add_argument("--type", default="toto", choices=["toto","titi"], help = "type (default: %(default)s)") source

June 24, 2017 · SergeM

Useful console commands

Some linux commands that I’ll probably need in the future User management Add user to a group sudo usermod -aG group user or sudo adduser <user> <group> Delete user userdel user Delete the user’s home directory and mail spool: userdel -r user Remove user from a group sudo gpasswd -d user group list all users: $ getent passwd list all groups: $ getent group list all groups of the current user:...

June 9, 2017 · SergeM

Markdown tricks

Expandable / collapsible code blocks in github The text is shown on click. 1 2 3 4 <details> <summary>Click to expand</summary> your hidden code </details> Results: Click to expand your hidden code Source

June 9, 2017 · SergeM

Streaming camera output from Raspberry Pi

Task was to transmit image from camera on raspberry pi through web interface to the PC. PC is connected to raspberry through local network. Enable camera on your raspberry pi in raspi-conf. Reboot. Save the following source code to a file on your raspberry (let’s say streaming.py) do sudo pip3 install picamera run python3 streaming.py on the target machine (PC) go to http://<your raspberry ip or host name>:8000. output from the camera must be streaming in realtime....

May 23, 2017 · SergeM

Rest API for TensorFlow model

TensorFlow Serving, sources - library for serving machine learning models. Written in C++ and Python. Server is in C++. Requires Bazel - Google’s build tool. Doesn’t work with python 3. Probably fast. TensorFlow: How to freeze a model and serve it with a python API Building a Machine Learning App with AWS Lambda (slides) Pipeline.io - End-to-End, Continuous Spark ML + Tensorflow AI Data Pipelines, Sources Interesting thread. They propose to use “saved_model_cli binary (in tools/), which you can feed a SavedModel, and pass input data via files....

May 21, 2017 · SergeM

Math explained and visualized

Here I will collect links to well-written articles about math and algorithms. Visual Information Theory. Nice post about probabilities, entropy. The whole blog has awesome visualizations. Highly recommended. MNIST For ML Beginners - Nice introduction into machine learning with TensorFlow Brilliantly wrong - blog by Alex Rogozhnikov, with fantastic visualizations of many algorithms and concepts Collection of data science notebooks. Deconvolution and Checkerboard Artifacts by Odena, et al., 2016 - explanation of deconvolution operation in neural networks with awesome interactive visualizations...

May 20, 2017 · SergeM

Crawling tools

Headless Chromium Splash - A javascript rendering service. guthub https://scrapy.org/ - An open source and collaborative framework for extracting the data you need from websites. Web crawler с использованием Python и Chrome

May 17, 2017 · SergeM