Charles Flèche's blog

Today I learned

This page contains short articles and small snippets on subjects I recently learned about.

Displaying live DNS queries

resolved monitor shows the live DNS queries. It needs to run as root.

$ resolvectl monitor

==== AUTHENTICATING FOR org.freedesktop.resolve1.subscribe-query-results ====
Authentication is required to subscribe query results.
Authenticating as: root
Password: 
==== AUTHENTICATION COMPLETE ==== Q: ssl.gstatic.com IN A
→ Q: ssl.gstatic.com IN AAAA
← S: success
← A: ssl.gstatic.com IN AAAA 2607:f8b0:4020:c07::5e
← A: ssl.gstatic.com IN A 142.250.69.67

→ Q: merino.services.mozilla.com IN A
→ Q: merino.services.mozilla.com IN AAAA
← S: success
← A: merino.services.mozilla.com IN CNAME mozilla.map.fastly.net
← A: mozilla.map.fastly.net IN AAAA 2a04:4e42:400::347
← A: mozilla.map.fastly.net IN AAAA 2a04:4e42::347
← A: mozilla.map.fastly.net IN AAAA 2a04:4e42:600::347
← A: mozilla.map.fastly.net IN AAAA 2a04:4e42:200::347
← A: mozilla.map.fastly.net IN A 151.101.1.91
← A: mozilla.map.fastly.net IN A 151.101.193.91
← A: mozilla.map.fastly.net IN A 151.101.65.91
← A: mozilla.map.fastly.net IN A 151.101.129.91

→ Q: incoming.telemetry.mozilla.org IN A
→ Q: incoming.telemetry.mozilla.org IN AAAA
← S: success
← A: incoming.telemetry.mozilla.org IN CNAME telemetry-incoming.r53-2.services.mozilla.com
← A: telemetry-incoming.r53-2.services.mozilla.com IN A 34.120.208.123
← A: r53-2.services.mozilla.com IN SOA ns-1507.awsdns-60.org awsdns-hostmaster.amazon.com 1 7200 900 1209600 86400 Q: telemetry-incoming.r53-2.services.mozilla.com IN A
→ Q: telemetry-incoming.r53-2.services.mozilla.com IN AAAA
→ C: incoming.telemetry.mozilla.org IN A
→ C: incoming.telemetry.mozilla.org IN AAAA
← S: success
← A: telemetry-incoming.r53-2.services.mozilla.com IN A 34.120.208.123
← A: r53-2.services.mozilla.com IN SOA ns-1507.awsdns-60.org awsdns-hostmaster.amazon.com 1 7200 900 1209600 86400 Q: 1.debian.pool.ntp.org IN AAAA
← S: success
← A: pool.ntp.org IN SOA d.ntpns.org hostmaster.pool.ntp.org 1783342806 5400 5400 1209600 3600 Q: 1.debian.pool.ntp.org IN A
← S: success
← A: 1.debian.pool.ntp.org IN A 162.159.200.1
← A: 1.debian.pool.ntp.org IN A 170.39.49.50
← A: 1.debian.pool.ntp.org IN A 216.232.132.95
← A: 1.debian.pool.ntp.org IN A 147.189.136.126

# ...

Changing several Blender scenes, single undo step

Text editors like VSCodium have an undo stack per opened text file. One can modify file A, switch to file B, edit it, get back to file A and trigger an undo: only the active file A will be reversed, not file B.

Blender can also modify several scenes within the same session, albeit with two major differences:

  1. a single document contains all scenes
  2. the user can edit a single scene at once

The second point is naturally enforced by the fact that switching the active scene is an undoable command by itself: select a scene, CTRL+Z, and we're back to the previous scene. In VSCodium selecting an active text editor panel goes into another undo stack, the one that allows to navigate through the documents, but not into the documents edition stack.

So I wanted to test my assumptions for a few situations regarding Python scripts and undo stack:

  1. Are all python changes revertable in a single undo ?
  2. Are the ops and data APIs handled differently ?
  3. What happens when a script modifies several scenes ?

TL;DR: python scripts changes are always revertable as a single undo step.

# Single scene, ops API

import bpy

for i in range(3):
    bpy.ops.mesh.primitive_cube_add(size=.5, location=(0, i, 0))

# RESULT: a single undo step removes all three cubes
# Single scene, data API

for i in range(3):
    obj = bpy.data.objects.new(name=f"empty_{i}", object_data=None)
    bpy.context.scene.collection.objects.link(obj)
#
# RESULT: a single undo step removes all three objects
# Several scenes, data API

def new(scene, name):
    for i in range(3):
        obj = bpy.data.objects.new(
            name=f"{scene.name}_{name}_{i}",
            object_data=None
        )
        scene.collection.objects.link(obj)

scene_a = bpy.data.scenes["SceneA"]
scene_b = bpy.data.scenes["SceneB"]

new(scene_a, "batch1")
new(scene_b, "batch2")
new(scene_b, "batch3")
new(scene_a, "batch4")

# RESULT: a single undo step removes all the objects created in the two scenes

Shared SdfLayer and TfNotice

What happens if two UsdStage in the same processus share an SdfLayer ? Turns out this does what is expected, with a twist:

  1. Modifying a stage modifies the other
  2. But the stage change notification is first sent to the last opened stage
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "usd-core>=26.5",
# ]
# ///

from pxr import Usd, UsdGeom, Sdf, Tf

stages = {}

def cb(notice, sender):
    print(f"{stages[sender]} changed")
l1 = Tf.Notice.Register(Usd.Notice.StageContentsChanged, cb, None)

sa = Usd.Stage.CreateInMemory()
stages[sa] = "StageA"

sb = Usd.Stage.Open(sa.GetRootLayer())
stages[sb] = "StageB"

print("Adding a prim to StageA")
sa.DefinePrim("/AddedFromStageA")

print("Adding a prim to StageB")
sb.DefinePrim("/AddedFromStageB")
Adding a prim to StageA
StageB changed
StageA changed
Adding a prim to StageB
StageB changed
StageA changed

UsdEditContext within an SdfChangeBlock

No USD level changes should be made within an SdfChangeBlock. I wanted to check if this still stands for "edition" that are purely at the USD level: as far as I know an changing a change edit target does not touch the Sdf API.

Turns out: StageEditTargetChanged are still emitted without throwing exception within an SdfChangeBlock.

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "usd-core>=26.5",
# ]
# ///

from pxr import Usd, UsdGeom, Sdf, Tf

def cb(notice, sender):
    print(notice)

s = Usd.Stage.CreateInMemory()

l1 = Tf.Notice.Register(Usd.Notice.StageEditTargetChanged, cb, s)
l2 = Tf.Notice.Register(Sdf.Notice.LayersDidChange, cb, None)

print("@@@ No SdfChangeBlock")

with Usd.EditContext(s, s.GetSessionLayer()):
    s.DefinePrim("/A_in_session")

with Usd.EditContext(s, s.GetRootLayer()):
    s.DefinePrim("/A_in_rootlayer")

print("@@@ Within an SdfChangeBlock")

with Sdf.ChangeBlock():
    lay = s.GetSessionLayer()
    with Usd.EditContext(s, lay):
        p = Sdf.CreatePrimInLayer(lay, "/B_in_session")
        p.specifier = Sdf.SpecifierDef

    lay = s.GetRootLayer()
    with Usd.EditContext(s, lay):
        p = Sdf.CreatePrimInLayer(lay, "/B_in_rootlayer")
        p.specifier = Sdf.SpecifierDef
@@@ No SdfChangeBlock
<pxr.Usd.Notice.StageEditTargetChanged object at 0x7fc4a8b7e680>
<pxr.Sdf.Notice.LayersDidChange object at 0x7fc4a8b7e680>
<pxr.Usd.Notice.StageEditTargetChanged object at 0x7fc4a8b7e680>
<pxr.Sdf.Notice.LayersDidChange object at 0x7fc4a8b7e680>
@@@ Within an SdfChangeBlock
<pxr.Usd.Notice.StageEditTargetChanged object at 0x7fc4a8b7e680>
<pxr.Usd.Notice.StageEditTargetChanged object at 0x7fc4a8b7e680>
<pxr.Sdf.Notice.LayersDidChange object at 0x7fc4a8b7e5f0>

Running rpi-imager as a normal user on debian / wayland

My debian workstation runs sway on wayland, but the latest rpi-imager is actually a X11 only AppImage that needs to run as root. To run it, I hence need to:

  • wrap it under policy kit authentication as no agent is running in the background
  • allow X11
  • pass some environment variables.

So:

xhost +SI:localuser:root
pkexec \
  env \
  DISPLAY=$DISPLAY \
  XAUTHORITY=$XAUTHORITY \
  QT_QPA_PLATFORM=xcb \
  rpi-imager

neovim shortcuts

I'm trying to learn neovim more thoroughly. It's heavily shortcut based that I need to learn little by little. Here is a list to help me memorize them:

Neovim Shortcuts Summary

Editing

  • D — Delete to end of line
  • d$ — Same as D
  • 3D — Delete to end of line, 3 lines
  • d0 — Delete backward to the start of the line
  • dw — Delete word forward
  • db — Delete word backward
  • dB — Delete word backward (incl. punctuation)
  • $dB — Delete last word of the line
  • ggVG — Select entire buffer
  • "+y — Yank selection to system clipboard
  • ggVG"+y — Select + copy whole buffer to clipboard

  • a start editing after the current cursor position

  • A start editing at the very end of the current line
  • i start editing on the current cursor position
  • I start editing at the first non-blank character of the current line
  • d start editing on a brand new line below the cursor
  • O start editing on a brand new line above the cursor.

Comments

  • Visual mode: gc toggle line comment
  • Visual mode: gb toggle block comments
  • Normal mode: gcc toggle comment on the current line
  • Normal mode: gbc toggle a block comment on the current line
  • Normal mode: gc + motion, comment out text defined by a motion

Indentation

  • Visual mode: > indent to the right
  • Visual mode: < indent to the left
  • Normal mode: >> indent to the right
  • Normal mode: << indent to the left

Windows / Splits

  • Ctrl-w v — Vertical split
  • Ctrl-w s — Horizontal split
  • :vsp filename — Vertical split with new file
  • :sp filename — Horizontal split with new file
  • :q / Ctrl-w q — Close current split
  • Ctrl-w c — Close current split
  • :close — Close current window (not last one)
  • :only / Ctrl-w o — Close all other splits

Tabs

  • :tabnew file — Open file in new tab
  • gt / gT — Next / previous tab
  • :tabclose — Close current tab
  • :tabs — List all tabs
  • 2gt — Jump to tab 2

LSP / Diagnostics

  • gd — Go to definition
  • gD — Go to declaration
  • gi — Go to implementation
  • gr — Go to references
  • K — Hover docs
  • <leader>rn — Rename symbol
  • <leader>ca — Code action
  • <C-k> — Signature help
  • <leader>e (custom) — Open diagnostic float
  • <F5> — DAP: continue
  • <F10> — DAP: step over
  • <F11> — DAP: step into
  • <leader>b (custom) — Toggle breakpoint
  • <leader>x (custom, trouble.nvim) — Toggle diagnostics panel
  • <leader>sS — Show workspace symbols
  • <leader>ss — Show document symbols

File browser

  • <leader>ff — Find files
  • <leader>fg — Live grep (requires rgrep)
  • <leader>fb — List buffers
  • <leader>fh — Help tags
  • <leader>fo — Recently opened files

Command-line / Ex Commands

  • :%s/.../.../g — Substitute over whole buffer
  • :g/^ /d — Delete all lines starting with two spaces
  • :g/^ / — Preview matches without deleting
  • :g/^ /normal dd — Delete matches via global+normal
  • :make — Run build (via makeprg)
  • :copen — Open quickfix list
  • :checkhealth nvim-treesitter — Check treesitter health
  • :checkhealth provider — Check clipboard provider

Misc

  • :qa — Quit vim and all buffers
  • <leader> — Custom prefix key, default \, commonly remapped to Space via vim.g.mapleader = " "

Removing a folder in every borg archive

A bunch of folders should not have been included into borg archive. Fortunately, borg allows to remove those is two steps:

# Assuming BORG_* environment variables are set

borg recreate --exclude 'path/to/folder1' --exclude 'path/to/folder2'
borg compact

A minimal debian for machinectl

I sometime need a quick, minimal debian server to test a package without compromising my main workstation. This machine has an uncommon networking setup, so I need systemd-resolved installed on the guest early.

# /etc/systemd/nspawn/my-container.nspawn
#
# This strip down sample shows the container network configuration.
# It needs to be named as the container. Here: my-container.nspawn

[Network]
VirtualEthernet=yes

[Exec]
ResolvConf=bind-uplink
# First, we create the minimal base system

sudo debootstrap                   \
  --variant=minbase                \
  --include=dbus,systemd-container \
  stable                           \
  /var/lib/machines/my-container   \
  https://deb.debian.org/debian/

# Then, we start the container and open a sheel

sudo machinectl start my-container
sudo machinectl shell my-container

# Lastly, we make sure the the networkd service is enabled and running

systemctl enable --now systemd-networkd

networkctl
IDX LINK  TYPE     OPERATIONAL SETUP     
  1 lo    loopback carrier     unmanaged
  2 host0 ether    routable    configured

2 links listed.

No roaming to activate Signal

I broke my phone and had to revive my old Cat S22, but I couldn't reactivate Signal: here in Canada I wouldn't receive the activation SMS sent to my French SIM. Turns out that roaming needs to be deactivated: Settings > Mobile Network > Roaming. I also disconnect the 4G for good measure, connected to WiFi and tried to activate my Signal account again. This time, it worked !

Presentation of Ynput's Ayon at the USDrinks

Today during the USD meeting at Moment Factory, Robin De Lillo shown us Ayon, an Open Source pipeline for VFX / animation / creative studios. It looks quite neat, and I like that they develop it in the open, fully on GitHub, with features funded by paying customers. I'd love to have a proper demo of their product some times in the future.

SIM management with mmcli

Broken phone, broken screen and a broken digitizer. It kept on registering random touches on the SIM PIN screen, unfortunately blocking it. My backup phone does not offer to enter the PUK PIN, so I had to find another way to unlock the SIM. Thankfully I have a Mobian compatible Pinephone where ModemManager could be installed.

$ sudo apt install modemmanager
$ sudo mmcli -i 0 --pin=<pin_number> --puk=<puk_number>

Tips courtesy of https://discourse.ubuntu.com/t/entering-sim-passwords/19909.

OpenDCC is released

The team behind the modular OpenUSD editor ShapeFX Loki released its Apache-2.0 licenced framework: OpenDCC.

From the README:

OpenDCC is an Apache 2.0 licensed, open-source Digital Content Creation (DCC) application framework for building modular, production-grade 3D tools. It combines a flexible, plugin-driven architecture with an industry-standard Qt-based interface, embedded Python scripting, OpenUSD and Hydra integration.

It looks very promising and I'm waiting for the first packages to try it.

ufbx, a single source Open Source C++ FBX loader

The FBX is a proprietary 3D scene file format from Autodesk. It is ubiquitous in the industry, especially in games. Its many iterations makes it a format rather difficult to work with and its closed source prevents free software application like Blender to directly link against the Autodesk libs: Blender's developpers had to reimplement parts of an FBX I/O themselves, with not great results.

Fortunately the ufbx seems to be a rather capable FBX reader. No writer yet, but Blender 4.5 will have a new ufbx based importer.

Testing USD hooks outside of Blender

Part of our pipeline is automated through a Blender extension that pre / post process USD with USDHooks. For a long time, our integration tests had to actually run the Blender executable: we had to import the USD modules, and those were not available through the pip installable bpy module. Installing usd-core alongside bpy wouldn't work: we would have two USD libraries running into the same process (one from usd-core, the other from bpy) and that's a recipe for disater.

Thankfully, since Blender 4.4, the function bpy.expose_bundled_modules() makes the third-party bundled modules, notably USD, available from the rest of the application.

import unittest

import bpy

bpy.expose_bundled_modules()

from pxr import Usd

class MyTest(unittest.TestCase):
    ...

Thanks to bpy.expose_bundled_modules() we can now run our tests fully from a Python interpreter, without running the Blender executable: - as we stay fully in the Python world, the setup is the same on local Windows workstation and Linux CI machines - we can simply run and debug our tests from the native IDEs integrations