OpenWrt x86 Build & QEMU Run — Full Setup Documentation ========================================================== This document describes the complete steps to build OpenWrt for x86_64 on Ubuntu 24.04 and run it inside QEMU with working networking, SSH, snapshots, and optional advanced features. In this section, you are going to learn .. panels:: :container: container pb-4 :column: col-lg-12 p-2 :card: shadow How to prepare an Ubuntu 24.04 host system for OpenWrt x86 build ? .. panels:: :container: container pb-4 :column: col-lg-12 p-2 :card: shadow How to configure and build OpenWrt for x86_64 architecture ? .. panels:: :container: container pb-4 :column: col-lg-12 p-2 :card: shadow How to generate OpenWrt x86 disk images (ext4, squashfs, GRUB, UEFI) ? .. panels:: :container: container pb-4 :column: col-lg-12 p-2 :card: shadow How to run OpenWrt x86 inside QEMU with networking, SSH, and snapshots ? .. panels:: :container: container pb-4 :column: col-lg-12 p-2 :card: shadow How to enable KVM acceleration, UEFI boot and router-style multi-NIC simulation ? .. panels:: :container: container pb-4 :column: col-lg-12 p-2 :card: shadow Topics in this section, * :ref:`Step 1: Host System Information ` * :ref:`Step 1.1: Technical Notes ` * :ref:`Step 2: Install Dependencies Required for OpenWrt Build ` * :ref:`Step 2.1: Technical Notes ` * :ref:`Step 3: Create Working Directory & Clone OpenWrt ` * :ref:`Step 3.1: Technical Notes ` * :ref:`Step 4: Configure OpenWrt for x86_64 Build ` * :ref:`Step 4.1: Technical Notes ` * :ref:`Step 5: Build OpenWrt ` * :ref:`Step 5.1: Technical Notes ` * :ref:`Step 6: Generated Build Artifacts ` * :ref:`Step 6.1: Technical Notes ` * :ref:`Step 7: Run OpenWrt in QEMU ` * :ref:`Step 7.1: Technical Notes ` * :ref:`Step 8: Optional - Run with Two NICs (LAN + WAN) ` * :ref:`Step 8.1: Technical Notes ` * :ref:`Step 9: Enable SSH Access ` * :ref:`Step 9.1: Technical Notes ` * :ref:`Step 10: Optional - Create QEMU Snapshots ` * :ref:`Step 10.1: Technical Notes ` * :ref:`Step 11: Optional - UEFI Boot ` * :ref:`Step 11.1: Technical Notes ` * :ref:`Step 12: Optional - Enable KVM Acceleration ` * :ref:`Step 12.1: Technical Notes ` * :ref:`Step 13: Resize Disk Image ` * :ref:`Step 13.1: Technical Notes ` * :ref:`Step 14: Troubleshooting ` * :ref:`Step 15: Advanced Full Router Simulation ` * :ref:`Step 15.1: Technical Notes ` .. _openwrt_x86_step1: .. _openwrt_x86_step1_1: .. tab-set:: .. tab-item:: Step 1: Host System Information The following host system was used for building: +----------------+------------------------------+ | Component | Value | +================+==============================+ | OS | Ubuntu 24.04.3 LTS | +----------------+------------------------------+ | Kernel | 6.8.0-86-generic | +----------------+------------------------------+ | Architecture | amd64 | +----------------+------------------------------+ | CPUs | 8 | +----------------+------------------------------+ | RAM | 7.3 GiB | +----------------+------------------------------+ | Disk Free | 337G | +----------------+------------------------------+ This configuration is more than sufficient for building OpenWrt. .. tab-item:: Step 1.1: Technical Notes - At least 4 cores and 4 GiB of RAM are recommended for a comfortable build experience. Builds will still work on smaller machines but will be slower. - The number of CPUs directly influences how many parallel jobs (``make -j$(nproc)``) you can safely run without thrashing memory or swap. - Disk usage for a full OpenWrt build (including sources, toolchain, and packages) can easily exceed 30–40 GiB. Ensure free space is available on the filesystem where ``~/openwrt-build`` resides. - If you intend to use KVM, verify that your CPU supports virtualization (Intel VT-x or AMD-V) and that it is enabled in BIOS/UEFI. - Use ``lscpu``, ``free -h`` and ``df -h`` to verify CPU, RAM and disk before starting long builds. .. _openwrt_x86_step2: .. _openwrt_x86_step2_1: .. tab-set:: .. tab-item:: Step 2: Install Dependencies Required for OpenWrt Build Run: .. code-block:: bash sudo apt update sudo apt install -y build-essential libncurses-dev gawk git gettext \ libssl-dev xsltproc unzip zip python3 python3-setuptools time \ zlib1g-dev subversion file flex bison qemu-system-x86 qemu-utils \ gperf autoconf automake libtool wget ccache (Optional) Faster rebuilds using ccache: .. code-block:: bash export CCACHE=1 .. tab-item:: Step 2.1: Technical Notes - ``build-essential`` provides the core toolchain (gcc, g++, make) required by the OpenWrt build system. - ``libncurses-dev`` is mandatory for the ``menuconfig`` TUI interface. - ``git`` and ``subversion`` are used to fetch the main OpenWrt tree and various package feeds. - ``gettext``, ``libssl-dev``, ``zlib1g-dev``, ``flex``, ``bison``, ``gperf``, ``autoconf``, ``automake``, ``libtool`` are used to build the cross-toolchain and many packages that OpenWrt depends on. - ``qemu-system-x86`` and ``qemu-utils`` are only needed on the *build host* for running the generated images; they are not build dependencies for the firmware itself. - ``python3`` and ``python3-setuptools`` are used by some build scripts and helper tools. - ``ccache`` caches compilations and is very useful when you repeatedly rebuild with small configuration changes. - After enabling ccache via ``export CCACHE=1``, you can monitor its stats with ``ccache -s``. - If you are running this on a minimal/stripped Ubuntu, verify that ``/usr/bin/python3`` exists and points to a supported Python version (3.8+ is generally fine). .. _openwrt_x86_step3: .. _openwrt_x86_step3_1: .. tab-set:: .. tab-item:: Step 3: Create Working Directory & Clone OpenWrt .. code-block:: bash mkdir -p ~/openwrt-build cd ~/openwrt-build git clone https://git.openwrt.org/openwrt/openwrt.git cd openwrt Update and install feeds: .. code-block:: bash ./scripts/feeds update -a ./scripts/feeds install -a .. tab-item:: Step 3.1: Technical Notes - Using a dedicated directory such as ``~/openwrt-build`` keeps the build tree isolated from other projects. - The default clone above pulls the current ``master`` (development) tree. - For a stable release, you can check out a specific branch or tag, e.g.: .. code-block:: bash git checkout v24.10.0 - ``./scripts/feeds update -a`` fetches all defined feed sources from ``feeds.conf.default`` (and ``feeds.conf`` if you created one). - ``./scripts/feeds install -a`` makes the packages from all feeds available in ``menuconfig`` and the build system. - If you add or remove feed sources later, rerun both of the above commands. - For reproducible builds, record the exact Git commit (``git rev-parse HEAD``) and keep it with your build notes. .. _openwrt_x86_step4: .. _openwrt_x86_step4_1: .. tab-set:: .. tab-item:: Step 4: Configure OpenWrt for x86_64 Build Start menuconfig: .. code-block:: bash make menuconfig Select the following: :: Target System → x86 Subtarget → x86_64 Target Profile → Generic Target Images: :: [*] ext4 [*] squashfs [*] Build GRUB images [*] Build GRUB EFI images [*] GZip images Filesystem: :: Root filesystem → ext4 Save and exit. .. tab-item:: Step 4.1: Technical Notes - ``Target System`` controls the CPU architecture and ABI. ``x86`` with ``x86_64`` subtarget generates 64-bit images suitable for most modern PCs and VMs. - ``Target Profile → Generic`` is a safe default for QEMU, VirtualBox, and most commodity x86 hardware. You can choose more specific profiles later if needed. - Enabling both ``ext4`` and ``squashfs`` images allows you to use a read-write ext4 root filesystem or a read-only squashfs with an overlay. - Enabling both ``Build GRUB images`` and ``Build GRUB EFI images`` allows the same build tree to produce Legacy BIOS and UEFI-bootable images. - The ``GZip images`` option produces compressed variants which are smaller but may add a small overhead during boot. - The selected root filesystem type (ext4 here) affects how you can resize partitions and how robust the system is to unclean shutdowns. - Your configuration is stored in ``.config`` at the root of the OpenWrt tree. Keep a backup of this file if you intend to reproduce the same image later: .. code-block:: bash cp .config ~/openwrt-x86_64.config .. _openwrt_x86_step5: .. _openwrt_x86_step5_1: .. tab-set:: .. tab-item:: Step 5: Build OpenWrt Start the full build: .. code-block:: bash make -j$(nproc) Typical build times: - First build: 1–2 hours - With ccache: 10–20 minutes .. tab-item:: Step 5.1: Technical Notes - ``make -j$(nproc)`` automatically uses all available CPU cores. On systems with limited RAM, you may want to lower this, e.g. ``make -j4``. - The first build downloads and builds the entire cross-toolchain, toolchain libraries, and packages. This is why it takes significantly longer than subsequent builds. - If the build fails, you can often re-run ``make`` without cleaning; the build system will pick up from where it stopped. - To completely clean the tree, you can use: .. code-block:: bash make clean # removes built images but keeps toolchain make dirclean # removes toolchain and most downloads make distclean # almost full reset (use with care) - Log output can be saved for analysis: .. code-block:: bash make -j$(nproc) V=s | tee build.log - The ``dl`` directory caches downloaded sources; do not remove it unless necessary, as it speeds up future builds. .. _openwrt_x86_step6: .. _openwrt_x86_step6_1: .. tab-set:: .. tab-item:: Step 6: Generated Build Artifacts Build output directory: :: ~/openwrt-build/openwrt/bin/targets/x86/64/ Key files: +---------------------------------------------+-------------------------------+ | File | Purpose | +=============================================+===============================+ | openwrt-x86-64-generic-ext4-combined.img | Main disk image for QEMU | +---------------------------------------------+-------------------------------+ | kernel.bin | Kernel image | +---------------------------------------------+-------------------------------+ | rootfs.tar.gz | Root filesystem | +---------------------------------------------+-------------------------------+ | \*.vdi OR \*.vmdk OR \*.vhdx | VM images | +---------------------------------------------+-------------------------------+ Copy the main image: .. code-block:: bash cp openwrt-x86-64-generic-ext4-combined.img ~/openwrt-fresh.img .. tab-item:: Step 6.1: Technical Notes - The exact filenames may include version strings or additional suffixes depending on your configuration, e.g. ``*-combined-efi.img`` for UEFI. - The ``*-ext4-combined.img`` image usually has both kernel and rootfs in a single disk image, suitable for direct use by QEMU. - ``kernel.bin`` and ``rootfs.tar.gz`` are useful if you want to assemble custom disk images or use alternative boot loaders. - Virtual disk formats (``*.vdi``, ``*.vmdk``, ``*.vhdx``) are typically generated if you enable the corresponding options in menuconfig. - Keeping a pristine copy such as ``~/openwrt-fresh.img`` is useful if you want a known-good starting point for experiments or snapshots. - Always verify the image type with ``file``: .. code-block:: bash file openwrt-x86-64-generic-ext4-combined.img .. _openwrt_x86_step7: .. _openwrt_x86_step7_1: .. tab-set:: .. tab-item:: Step 7: Run OpenWrt in QEMU Basic run command: .. code-block:: bash qemu-system-x86_64 \ -m 1024 -smp 2 \ -drive file=openwrt-fresh.img,if=virtio,format=raw \ -nic user,model=virtio,hostfwd=tcp::2222-:22 Default OpenWrt login: :: root (no password) Default networking: - ``br-lan`` → 192.168.1.1 - QEMU NAT → 10.0.2.x .. tab-item:: Step 7.1: Technical Notes - ``-m 1024`` allocates 1 GiB RAM to the VM. You can reduce to 512 MiB for lighter lab setups, or increase for heavier workloads. - ``-smp 2`` gives the VM 2 virtual CPUs. Many OpenWrt tasks do not require many cores, but additional vCPUs help for routing/firewalling benchmarks. - ``if=virtio`` and ``model=virtio`` give paravirtualized disk and NIC devices, which are faster and better supported than legacy emulated devices like e1000. - ``-nic user,...,hostfwd=tcp::2222-:22`` uses QEMU’s user-mode networking and forwards host TCP port 2222 to guest port 22 (SSH). - If you want a serial console instead of a graphical window, you can add: .. code-block:: bash -nographic -serial mon:stdio - When running multiple concurrent VMs, ensure each uses a unique host port (e.g. 2222, 2223, ...). .. _openwrt_x86_step8: .. _openwrt_x86_step8_1: .. tab-set:: .. tab-item:: Step 8: Optional - Run with Two NICs (LAN + WAN) Useful for router/firewall testing. .. code-block:: bash qemu-system-x86_64 \ -m 1024 -smp 2 \ -drive file=openwrt-fresh.img,if=virtio \ -device virtio-net-pci,netdev=lan \ -netdev user,id=lan,hostfwd=tcp::2222-:22 \ -device virtio-net-pci,netdev=wan \ -netdev user,id=wan OpenWrt NIC mapping: - ``eth0`` → LAN - ``eth1`` → WAN (DHCP from QEMU) .. tab-item:: Step 8.1: Technical Notes - QEMU creates two separate networks: - ``lan``: user-mode network with port forwarding to 2222. - ``wan``: a second user-mode network that typically gets its own NATed range and DHCP server. - OpenWrt will usually bridge ``eth0`` into ``br-lan``, while ``eth1`` is treated as a WAN interface with DHCP client enabled. - This setup mimics a typical home router: LAN behind the router, WAN connected to an upstream NAT. - For advanced simulations, replace ``-netdev user,id=wan`` with a tap device or Linux bridge to connect the WAN side to real networks. .. _openwrt_x86_step9: .. _openwrt_x86_step9_1: .. tab-set:: .. tab-item:: Step 9: Enable SSH Access SSH into OpenWrt from host: .. code-block:: bash ssh -p 2222 root@localhost .. tab-item:: Step 9.1: Technical Notes - OpenWrt’s default SSH server is ``dropbear``. It listens on port 22 on the LAN side by default. - On a fresh image, the ``root`` account usually has no password. The first thing you should do is set one: .. code-block:: bash passwd - If you cannot connect, check from inside OpenWrt: .. code-block:: bash netstat -tlnp | grep 22 logread -e dropbear - If you remove or disable SSH access via firewall or system config, the ``hostfwd`` option in QEMU will still listen, but connections will fail immediately or timeout. - For key-based authentication, copy your public key into ``/etc/dropbear/authorized_keys``. .. _openwrt_x86_step10: .. _openwrt_x86_step10_1: .. tab-set:: .. tab-item:: Step 10: Optional - Create QEMU Snapshots Create snapshot: .. code-block:: bash qemu-img snapshot -c clean ~/openwrt-fresh.img Restore: .. code-block:: bash qemu-img snapshot -a clean ~/openwrt-fresh.img .. tab-item:: Step 10.1: Technical Notes - Snapshots should be created while the VM is powered off or after a clean shutdown to avoid filesystem inconsistencies. - ``snapshot -c NAME`` creates an internal snapshot stored in the same image file. Not all formats support snapshots equally; qcow2 is generally more snapshot-friendly than raw. - To list snapshots: .. code-block:: bash qemu-img snapshot -l ~/openwrt-fresh.img - For heavy experimentation, consider converting the image to qcow2 before snapshotting: .. code-block:: bash qemu-img convert -O qcow2 openwrt-fresh.img openwrt-fresh.qcow2 - Snapshots are not a backup replacement. Keep separate copies of known-good images as well. .. _openwrt_x86_step11: .. _openwrt_x86_step11_1: .. tab-set:: .. tab-item:: Step 11: Optional - UEFI Boot .. code-block:: bash qemu-system-x86_64 \ -m 1024 \ -smp 2 \ -drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE.fd \ -drive if=pflash,format=raw,file=/usr/share/OVMF/OVMF_VARS.fd \ -drive file=openwrt-fresh.img,if=virtio .. tab-item:: Step 11.1: Technical Notes - UEFI boot typically requires the ``OVMF`` firmware, provided by packages such as ``ovmf`` or ``qemu-ovmf`` on many distributions. - ``OVMF_CODE.fd`` is the read-only code image; ``OVMF_VARS.fd`` holds modifiable NVRAM/UEFI variables. - Keep a backup copy of ``OVMF_VARS.fd`` if you want to restore a clean UEFI environment later. - Ensure you built UEFI-compatible images in OpenWrt (GRUB EFI images enabled in menuconfig). - Secure Boot is generally not enabled in this simple UEFI setup; if you emulate Secure Boot, additional signing steps are required. .. _openwrt_x86_step12: .. _openwrt_x86_step12_1: .. tab-set:: .. tab-item:: Step 12: Optional - Enable KVM Acceleration For significantly faster VM execution: .. code-block:: bash qemu-system-x86_64 \ -enable-kvm -cpu host \ -m 1024 -smp 2 \ -drive file=openwrt-fresh.img,if=virtio \ -nic user,model=virtio,hostfwd=tcp::2222-:22 .. tab-item:: Step 12.1: Technical Notes - KVM requires: - Hardware virtualization support (Intel VT-x or AMD-V). - ``kvm`` kernel modules loaded (``kvm_intel`` or ``kvm_amd``). - You can verify availability with: .. code-block:: bash lsmod | grep kvm [ -r /dev/kvm ] && echo "KVM available" - ``-cpu host`` exposes the host CPU features directly to the guest, enabling better performance and instruction set usage. - If you get a permission error on ``/dev/kvm``, add your user to the ``kvm`` group and re-login: .. code-block:: bash sudo usermod -aG kvm "$USER" - Without ``-enable-kvm``, QEMU falls back to pure software emulation, which can be significantly slower. .. _openwrt_x86_step13: .. _openwrt_x86_step13_1: .. tab-set:: .. tab-item:: Step 13: Resize Disk Image Increase size to 2GB: .. code-block:: bash qemu-img resize openwrt-fresh.img 2G Inside OpenWrt: .. code-block:: bash resize2fs /dev/sda2 .. tab-item:: Step 13.1: Technical Notes - Always shut down the VM cleanly before resizing the image file to avoid corruption. - ``qemu-img resize`` changes the size of the virtual disk, but not the partition table or filesystem inside the guest. - OpenWrt’s default partition layout usually has the root filesystem on ``/dev/sda2`` (for combined ext4 images), but confirm with: .. code-block:: bash lsblk fdisk -l /dev/sda - If there is unallocated space after the root partition, you may need to grow the partition first (using tools like ``fdisk`` or ``parted``) and then run ``resize2fs``. - For squashfs-based images, root is read-only and not directly resizable; ext4 images are better for experiments that need more writable space. .. _openwrt_x86_step14: .. tab-set:: .. tab-item:: Step 14: Troubleshooting **No Networking** Check device list: .. code-block:: bash ip a logread -e eth - Ensure that expected interfaces (e.g. ``eth0``, ``eth1``) are present and have the correct MAC addresses as per your QEMU command line. - Check network configuration files: .. code-block:: bash cat /etc/config/network - ``logread -e eth`` filters system log entries containing “eth”, which helps identify link up/down events or driver issues. - From the host side, re-check QEMU options (``-nic``, ``-netdev``) for typos or conflicting settings. **OpenWrt Stuck at GRUB** Disable EFI in menuconfig: :: Target Images → Uncheck EFI image - If the image was built only for EFI but QEMU is configured for legacy BIOS (or vice versa), GRUB or firmware may hang. - Double-check that your QEMU boot configuration matches what you built (BIOS vs UEFI). - For debugging, you can enable GRUB’s console and edit the boot entry at the GRUB menu to see more detailed messages. - If the disk image is corrupted, try booting with a backup image or run ``fsck`` from a rescue ISO. **SSH Not Working** Validate firewall rules: .. code-block:: bash uci show firewall | grep ssh - Verify that ``dropbear`` is enabled: .. code-block:: bash /etc/init.d/dropbear status - Make sure SSH is allowed on the desired zone (typically LAN). Example rule snippet: .. code-block:: bash uci show firewall | grep 22 - If you changed the SSH port, ensure your host-side port forwarding and firewall configuration match the new port. - Use ``logread -e dropbear`` to see authentication and connection errors. - From the host, test basic connectivity with: .. code-block:: bash telnet localhost 2222 .. _openwrt_x86_step15: .. _openwrt_x86_step15_1: .. tab-set:: .. tab-item:: Step 15: Advanced Full Router Simulation .. code-block:: bash qemu-system-x86_64 \ -enable-kvm -cpu host \ -m 2048 -smp 4 \ -drive file=openwrt-fresh.img,if=virtio \ -device virtio-net-pci,netdev=lan \ -netdev user,id=lan,hostfwd=tcp::2222-:22 \ -device virtio-net-pci,netdev=wan \ -netdev tap,id=wan,ifname=tap0,script=no,downscript=no .. tab-item:: Step 15.1: Technical Notes - This configuration gives OpenWrt: - A LAN interface (``lan``) using QEMU user-mode networking with SSH port forwarding. - A WAN interface connected to a host tap interface (``tap0``), which you can attach to a Linux bridge or physical NIC. - On the host, you can create and configure ``tap0`` as follows: .. code-block:: bash sudo ip tuntap add dev tap0 mode tap sudo ip link set tap0 up # Optionally bridge with another interface: # sudo ip link add name br0 type bridge # sudo ip link set tap0 master br0 # sudo ip link set eth0 master br0 - This setup lets you route real traffic through the OpenWrt VM and test firewall, NAT, QoS, and VPN configurations as if it were a physical router. - Be cautious when bridging to your real network; misconfiguration can affect connectivity of your host or other machines on the LAN. - For repeatable lab scenarios, consider scripting the tap/bridge setup and teardown, and documenting IP addressing schemes used on both host and VM.