Posts tagged with "python"

Python is a general purpose programming language, used for a variety of tasks across a number of industries.

In recent years Python has become increasingly popular due to it's good cross platform support combined with applicability to tasks in a number of areas:

  • Automation (build pipelines, glue code scripting, pywin32, excel integrations, etc)
  • Data science (Jupyter-notebooks, pandas, SQL, HD5 etc)
  • Machine learning (Scikit-learn, Tensorflow, pytorch, Keras, etc)
  • Web apps (via WSGI/ASGI and web frameworks such as Django, Flask, Quart, Pyramid, etc)
  • Network automation (netmiko, requests, fabric, ansible, connexion, textFSM, asyncio, etc)
  • GIS (geopandas, xarray, folium, etc)

25 Posts

Python's data model and operators

I saw this post over on Linkedin: I think what is seen to be intuitive will depend heavily on how much the people reading the code are…

Published on July 11th, 2020 by Janis Lesinskis.

Updating your files to the newer format

One of the not so nice things about legacy installable python packages is the executable nature of files. This means amongst other things that you can't reliably determine the contents of some files without arbitrary code execution. This is what led to PEP 518 which specified a declarative metadata format that can be stored in easy to parse file. Updating is a pain though, but thankfully there's some tooling that makes this easier as I found out recently.

Published on May 11th, 2020 by Janis Lesinskis.

Running pip as a Python module

Since Python 3.4 you have been able to run pip as though it were a Python module, this actually prevents a variety of annoying issues from occurring.

Published on November 3rd, 2019 by Janis Lesinskis.

Different ways to drop into an interactive Python shell

When you are developing in Python it can be handy to drop into an interactive Read Evaluate Print Loop (REPL) shell. Being able to drop into…

Published on July 29th, 2019 by Janis Lesinskis.

Forwards compatibility with Python generators

PEP 479 changed how Python handled generators in a backwards incompatible manner, here's how to get compatible code working from Python 3.5+

Published on July 3rd, 2019 by Janis Lesinskis.

A day in the life of a package maintainer

I was bumping some versions for the Persephone project today to try remove some security issues from the project dependencies. Package…

Published on June 28th, 2019 by Janis Lesinskis.

Deprecating module level variables

A good strategy when managing your packages is to have a strategy for deprecating functionality across versions. This way developers get…

Published on June 27th, 2019 by Janis Lesinskis.

Making Raw NBConvert cells look visually distinct

When using Jupyter notebooks have you found the styling of code cells to be too visually similar to Raw NBConvert cells? We found the similarities in formatting made it visually harder to see when a cell was set to the wrong cell type when we were running workshops and editing our own data science notebooks. Here's how we changed it.

Published on March 27th, 2019 by Janis Lesinskis.

How to configure Jupyter to not autocomplete

Have you ever found the autocompletion of certain characters like brackets or quotes to be an annoyance in Jupyter? If so here's how to change the configuration to disable this.

Published on March 24th, 2019 by Janis Lesinskis.

How to capture the standard output stream from a Jupyter notebook cell

One thing you may want to do is to capture all of the standard output stream from a cell in a Jupyter notebook. You might also not want to change your code, here's how you can do that...

Published on March 9th, 2019 by Janis Lesinskis.

How to work with Jupyter notebook cell input and output

Wondered what those and things are in your Jupyter notebooks and how to make use of them?

Published on March 8th, 2019 by Janis Lesinskis.

A hello-world with Python packaging

When you have used Python a lot you'll notice that you will use tools such as you are installing some python code in a way that can be…

Published on December 17th, 2018 by Janis Lesinskis.

CPython internals talk at Melbourne Python Users Group

On Thursday I presented a talk about how CPython manages memory at the Melbourne Python Users Group. After introducing the notion of a…

Published on November 17th, 2018 by Janis Lesinskis.

NumPy introduction

You want to deal with bulk data from your Python program. You realise that looping over every cell of a huge array from your Python code would be silly. You also would like the convenience of many kinds of canned routine to transform your data easily and efficiently. Enter NumPy!

Published on September 16th, 2018 by Nick Downing.

Finding the size of Python Objects

Ever wondered exactly much memory in bytes an arbitrary Python object requires?

Published on September 4th, 2018 by Janis Lesinskis.

A couple of questions about mypy

Last night Robbie Clarken presented a great talk about Mocking and how it fits in with testing in general. There were a few questions he had over email about how mypy could be used in addition to Mocking to get better coverage of issues with parameters in function calls being tested. Seeing as this is helping him with a real process we agreed that there's value in blogging this so a wider audience can get exposure to these techniques.

Published on July 20th, 2018 by Janis Lesinskis.

Python's __hash__ function is not a cryptographic hash

You may have noticed that Python has an internal hash method, here’s why it is not suitable as a cryptographic hash and some suggestions for alternatives if you do need a cryptographic hash.

Published on July 14th, 2018 by Janis Lesinskis.

Tensorflow serving now officially supports python3

Recently we have been using a lot of Tensorflow in projects. As far as libraries go it's really quite nice, however it's not a native Python…

Published on June 26th, 2018 by Janis Lesinskis.

Limiting the runtime of Python unit tests

Ever needed to limit how long your unit tests can run for? Here's an easy way to do that with pytest. There's a package called pytest…

Published on June 23rd, 2018 by Janis Lesinskis.

Cache locality in Python lists

The other day I got asked what the cache locality of lists in CPython was and how the lists are layed out in memory. Because I always tend…

Published on June 18th, 2018 by Janis Lesinskis.

Pytest fixtures with Flask

Recently I've been working on a web API with Flask, here's how I went about making unit tests with pytest work well with Flask and testing the API via requests.

Published on May 27th, 2018 by Janis Lesinskis.

Enforcing foreign key constraints with SQLite

There was a perplexing issue we ran into when testing the Persephone web API. There was a foreign key constraint specified in SQLAlchemy but you could add in new entries with bogus foreign keys without the DB engine raising an . Why did this happen?

Published on May 7th, 2018 by Janis Lesinskis.

Associative mappings in language design

We have been working on projects in JavaScript and Python a fair bit lately. In the past we ported a few legacy PHP apps over to newer…

Published on April 23rd, 2018 by Janis Lesinskis.

Python self references

This is a bit of Python trivia but Python containers can store items that are self references.

Published on April 22nd, 2018 by Janis Lesinskis.

Logging uncaught exceptions in Python applications

You want to be able to get good logging data from your applications to be able to reduce the amount of time that you require in order to fix issues and debug problems. This is especially important for any unhandled exceptions that crash the program. This tutorial will show you a convenient way of making sure these are logged.

Published on February 6th, 2018 by Janis Lesinskis.