Python

[asyncio](https://docs.python.org/3/library/asyncio.html) is a library to write concurrent code using the async and await syntax.

asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc.

asyncio is often a perfect fit for IO-bound and high-level structured network code.

asyncio provides a set of high-level APIs to:

• run Python coroutines concurrently and have full control over their execution
• perform network IO and IPC
• control subprocesses
• distribute tasks via queues
• synchronize concurrent code

1.Runners

asyncio.run(coro, \*, debug=None) Execute the coroutine coro and return the result.

This function runs the passed coroutine, taking care of managing the asyncio event loop, finalizing asynchronous generators, and closing the threadpool.

This function cannot be called when another asyncio event loop is running in the same thread.

If debug is True, the event loop will be run in debug mode. False disables debug mode explicitly. None is used to respect the global Debug Mode settings.

This function always creates a new event loop and closes it at the end. It should be used as a main entry point for asyncio programs, and should ideally only be called once.

Example:

async def main(): await asyncio.sleep(1) print('hello')

asyncio.run(main())

• [Subprocess](subprocess.md)

Subprocesses are separate instances of a program that run alongside the main program. They can be used to perform tasks in parallel, divide a large task into smaller pieces, or run separate programs that interact with the main program.

When a subprocdss has finished running then is it typically returning an integer. The integer is usually "0" when the process has finished successfully and an other number when it has failed.

[htop](https://htop.dev/) is command line tool for inspecting processes.

The python [subprocess module](https://docs.python.org/3/library/subprocess.html) allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. Most of your interaction with the Python subprocess module will be via the run() function. This blocking function will start a process and wait until the new process exits before moving on.

There are a few common reasons why you might want to call the shell with the Python subprocess module:

• When you know certain commands are only available via the shell, which is more common on Windows.
• When you’re experienced in writing shell scripts with a particular shell, so you want to leverage your ability there to do certain tasks while still working primarily in Python.
• When you’ve inherited a large shell script that might do nothing that Python couldn’t do, but would take a long time to reimplement in Python.

Common reasons for using subprocess itself are similar in nature to using the shell with subprocess:

• When you have to use or analyze a black box, or even a white box.
• When you want a wrapper for an application.
• When you need to launch another application.
• As an alternative to basic shell scripts

It is possible to divide up the different parts of the command in a list. For example, to [open](https://ss64.com/osx/open.html) TextEdit on a Mac:

import subprocess

subprocess.run(["open", "-a", "TextEdit"])

Calling run() isn’t the same as calling programs on the command line. The run() function makes a [system call](https://en.wikipedia.org/wiki/System_call), foregoing the need for a shell. You’ll cover interaction with the shell in a later section.

<span class="underline">Learning Resources on Python Subprocess</span>

• [RealPython](https://realpython.com/python-subprocess/)

• Splitting up a command with shlex

  If you are unsure on how to split up a command then can you use shlex for help. Bear in mind that shlex is designed for POSIX compliant systems and may not work well in Windows environments.

  import shlex

  shlex.split("echo 'Hello, World!'") # Returns ['echo', 'Hello, World!']

  You’ll note that the message, which contains spaces, is preserved as a single token, and the extra quotation marks are no longer needed. The extra quotation marks on the shell serve to group the token together, but since subprocess uses sequences, it’s always unambiguous which parts should be interpreted as one token.

• Dealing with Errors from Subprocesses

  When you use run(), the return value is an instance of the CompletedProcess class. As the name suggests, run() returns the object only once the child process has ended. It has various attributes that can be helpful, such as the args that were used for the process and the returncode.

  To deal with subprocess exceptions use the try&#x2026;except statments.

  try: subprocess.run( ["python", "timer.py", "5"], timeout=10, check=True ) except FileNotFoundError as exc: print(f"Process failed because the executable could not be found.\n{exc}") except subprocess.CalledProcessError as exc: print( f"Process failed because did not return a successful return code. " f"Returned {exc.returncode}\n{exc}" ) except subprocess.TimeoutExpired as exc: print(f"Process timed out.\n{exc}")

  1. The CompletedProcess Object

     To see this clearly, you can assign the result of run() to a variable, and then access its attributes such as .returncode.

     import subprocess

     completed_process = subprocess.run(["ping", "-c 1", "google.com"]) print(f"\nProcess return code is:\t {completed_{process.returncode}}")

     Usually you want to know if the subprocess has failed. You can do this with the `check=True` flag.

     completed_process = subprocess.run(["ping", "-c 1", "example.text"], check=True)

  2. Processes Which Never Finish

     Processes which never complete will not necessarily raise any error messages. The way to deal with them is to set a limited amount of time to way for them with the `timeout​=[number]` parameter.

     subprocess.run(["sleep", "100"], timeout=10)

  3. FileNotFoundError for Programs That Don’t Exist

     Is raised if you try and call a program that doesn’t exist on the target system.

     subprocess.run(["not_aprogram"]) # Returns FileNotFoundError: The system cannot find the file specified

     This type of error is raised no matter what, so you don’t need to pass in any arguments for the FileNotFoundError.

• Working with Text Based Programs

  There are two separate processes that make up the typical command-line experience.

  1. The interpreter, which is typically thought of as the whole CLI. Common interpreters are Bash on Linux, Zsh on macOS, or PowerShell on Windows.
  2. The interface, which displays the output of the interpreter in a window and sends user keystrokes to the interpreter. The interface is a separate process from the shell, sometimes called a terminal emulator.

  The run() function can make a system call directly and doesn’t need to go through the shell to do so. Many programs that are thought of as shell programs, such as [Git](git.md), are really just text-based programs that don’t need a shell to run. This is especially true of UNIX environments, where all of the familiar utilities like `ls`, `rm`, `grep`, and `cat` are actually separate executables that can be called directly.

  Using the flag `shell​=True` uses the arguments `["sh", "-c"]` in the background. It means that these two versions of run() are interchangeable, either a) `run(["sh", -c", "a long command"])` or b) `run(["a long command"], shell=True)`.

  1. Capturing output

     To capture output when using the subprocess.run() command you can use the `capture<sub>output</sub>=True`. You then need to specify which stream of data you want to show, like `stdout` or `stderr`.

     number = subprocess.run("echo $RANDOM", capture_output=True, shell=True) number.stdout # Returns b'32247\n'

     You will receive the output in bytes which you will then have to decode. To simplify this can you specify the encoding format with the parameter with the `encoding` parameter. For example to use `encoding​="utf-8"`. Since you then get a string back can you for example use strip() to remove newlines.

     number = subprocess.run("echo $RANDOM", capture_output=True, shell=True, encoding="utf-8") number.stdout.strip() # Returns '32247'

  2. Giving input to subprocess

     Use the `inputer​="something"` paramter to give input to the subprocess. Using the encoding parameter puts the run)() command into text mode. Note that the imput an be "stacked" into the stream.

     Small reaction time game.

     from time import perf_counter, sleep from random import random

     print("Press enter to play") input() print("Ok, get ready!") sleep(random() * 5 + 1) print("go!") start = perf_counter() input() end = perf_counter() print(f"You reacted in {(end - start) * 1000:.0f} milliseconds!\nGoodbye!")

     Stacking two "return" on top of each other as input.

     import subprocess process = subprocess.run(["python", "reaction_game.py"], input="\n\n", encoding="utf-8")

     Outputs the following.

     Press enter to play Ok, get ready! go! You reacted in 0 milliseconds! Goodbye!

  3. Pipes and the Shell

     The Python subprocess uses pipes extensively to interact with the processes that it starts. When using the parameter `capture<sub>output</sub>​=True` is Python doing this for us. The following two statments are equivalent:

     number = subprocess.run(["python", "number_generator.py"], capture_output=True)

     Is the same as:

     number = suprocess.run(["python", "number_generator.py"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

     Pipe buffers have limited capacity and if you run out of it, then can you create a file to hold the data instead.

     from tempfile import TemporaryFile with TemporaryFile() as f: ls_process = subprocess.run(["python", "magic_number.py"], stdout=f) f.seek(0) print(f.read().decode("utf-8"))

     However, it is not possible to pass a bytes object or a string directly to the stdin argument. It needs to be something file-like.

     To link up two processes with a pipe from within subprocess is something that you can’t do with run(). Instead, you can delegate the plumbing to the shell.

     If you’re on a UNIX-based system where almost all typical shell commands are separate executables, then you can just set the input of the second process to the .stdout attribute of the first CompletedProcess:

     import subprocess ls_process = subprocess.run(["ls", "/usr/bin"], stdout=subprocess.PIPE) grep_process = subprocess.run( ["grep", "python"], input=ls_{process.stdout}, stdout=subprocess.PIPE ) print(grep_{process.stdout.decode}("utf-8"))

     Here the .stdout attribute of the CompletedProcess object of ls is set to the input of the grep_process. It’s important that it’s set to input rather than stdin. This is because the .stdout attribute isn’t a file-like object.

     As an alternative, you can operate directly with files too, setting them to the standard stream parameters. When using files, you set the file object as the argument to stdin, instead of using the input parameter:

     import subprocess from tempfile import TemporaryFile with TemporaryFile() as f: ls_process = subprocess.run(["ls", "/usr/bin"], stdout=f) f.seek(0) grep_process = subprocess.run( ["grep", "python"], stdin=f, stdout=subprocess.PIPE )

     0 # from f.seek(0) print(grep_{process.stdout.decode}("utf-8"))

  4. Popen

     The [Popen()](https://docs.python.org/3/library/subprocess.html#popen-objects) constructor is very similar in appearance to using `run()`. If there’s an argument that you can pass to run(), then you’ll generally be able to pass it to Popen(). The fundamental difference is that it’s not a [blocking](https://en.wikipedia.org/wiki/Blocking_(computing)) call—rather than waiting until the process is finished, it’ll run the process in parallel.

     A key point to note is that in contrast to run(), which returns a CompletedProcess object, the Popen() constructor returns a Popen object. The standard stream attributes of a CompletedProcess point to bytes objects or strings, but the same attributes of a Popen object point to the actual streams. This allows you to communicate with processes as they’re running.

     The Popen class has several methods that allow you to interact with the process, such as communicate(), poll(), wait(), terminate(), and kill().

     For example, run a command, capture the output and print the output (if any):

     process = subprocess.Popen(['ls', '-la'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) output, error = process.communicate()

     print(output.decode('utf-8')) print(error.decode('utf-8'))

     The `communicate()` method to retrieve the output and error (if any) from the subprocess. The \`output\` and \`error\` variables are bytes objects, so we use the `decode()` method to convert them to a string before printing them.

  5. Poll with Popen

     To deal with the non-blocking nature of Popen, do use `poll()`.

     import subprocess from time import sleep

     with subprocess.Popen( ["python", "timer.py", "5"], stdout=subprocess.PIPE ) as process:

     def poll_andread(): print(f"Output from poll: {process.poll()}") print(f"Output from stdout: {process.stdout.read1().decode('utf-8')}")

     poll_andread() sleep(3) poll_andread() sleep(3) poll_andread()

     This program calls the timer process in a context manager and assigns stdout to a pipe. Then it runs the .poll() method on the Popen object and reads its stdout.

     The .poll() method is a basic method to check if a process is still running. If it is, then .poll() returns None. Otherwise, it’ll return the process’s exit code.

     Then the program uses .read1() to try and read as many bytes as are available at .stdout.

     If you put the Popen object into text mode and then call .read() on .stdout, the call to .read() would be blocking until it reached a newline.

     To read as many bytes as are available at that time, disregarding newlines, you need to read with .read1(). It’s important to note that .read1() is only available on byte streams, so you need to make sure to deal with encodings manually and not use text mode.

  6. Linking Processes with PIPE

     As an example, are two processes started in parallel. They are joined with a common pipe, and the for loop takes care of reading the pipe at stdout to output the lines.

     ls_process = subprocess.Popen(["ls", "-la"], stdout=subprocess.PIPE) grep_process = subprocess.Popen( ["grep", "Movies"], stdin=ls_{process.stdout}, stdout=subprocess.PIPE )

     for line in grep_{process.stdout}: print(line.decode("utf-8").strip())

We import a module named datetime to work with dates and time as date/time objects.

Print the datetime of now() as year-month-day-hour-minute-second.

from datetime import datetime

date_object = datetime.now()

print(date_{object.strftime}("%Y-%m-%d-%H-%M-%S"))

[strftime](https://strftime.org/) is a method in the Python datetime module that stands for "string format time". It is used to convert a datetime object into a string of a specified format. The method takes a string format parameter that specifies the format of the output string. The method returns a string representing the date and time in the specified format.

A format code is a string with a bunch of special tokens that’ll be replaced with information from the datetime object.

If you want to represent your datetime objects in completely unambiguous terms, then you’ll first need to make your object time zone aware. Once you have a time zone–aware object, the time zone information gets added to your ISO timestamp.

now = datetime.now()

print(now.tzinfo) None

now_aware = now.astimezone()

print(now_aware.tzinfo) Romance Standard Time

now_aware.tzinfo datetime.timezone(datetime.timedelta(seconds=3600), 'Romance Standard Time')

now_{aware.isoformat}() '2022-11-22T14:31:59.331225+01:00'

You can also perform arithmetic operations on datetime objects like addition and subtraction.

next_day = now + datetime.timedelta(days=1)
previous_day = now - datetime.timedelta(days=1)

Use the `input()` function to ask the user for input.

input_fromuser = input("Give me some input: ") print(input_fromuser)

The input function always returns a value of type String. You can convert the input from the user to another type through [type casting](python-types.md).

age = int(input("Give me your age: ")) print(f"This is your age in 50 years: {age + 50}")

Use the print() function to send output to the console. print() takes a few arguments:

• sep= : Defaults to a whitespace (" ")

• end= : Defaults to a newline ("\n")

  age = int(input("Give me your age: ")) print("This is your age today: " + str(age), "This is your age in 50 years: " + str(age + 50), sep="\n->", end="\n\n\n\n")

Do exec(open("filename.py").read()) you want to execute a python module/script from a REPL.

If you just want to run the file can you also import the whole file and then after you have updated the file can you get the updated version with reload file.