[cfe-dev] Clang builds a working Linux Kernel (Boots to RL5 with SMP, networking and X, self hosts)

Name: [cfe-dev] Clang builds a working Linux Kernel (Boots to RL5 with SMP, networking and X, self hosts)
Rating: 4.7 (8127 reviews)

Mon Oct 25 01:28:50 CDT 2010

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Clang can now compile a functional Linux Kernel (version 2.6.36, SMP). General Details =============== * Development and testing environment is a Macbook 5.1 (Intel C2D, x86_64) running Debian GNU/Linux. * The kernel can successfully boot to runlevel 5 (aka X + networking) on the Macbook, both on bare metal and in Qemu. * The kernel can successfully boot to runlevel 3 on a secondary test machine, a microATX desktop box (Intel Atom). I haven't tried to start X on this box yet. * The kernel can self-host; I am currently running a "fourth generation" self- hosted Linux kernel built by a "fourth generation" Clang. Major Subsystems Successfully Built =================================== I have likely left out a few things here. * Core kernel stuff, Filesystems, Bus, PCI jazz, ACPI - No problems here that I've run into. I'm sure there are issues under the hood here, just haven't gotten to stress testing these yet. * SMP, SMT, SysV, pthreads, and POSIX IPC - I have tested these fairly extensively, (using the tiobench and rt-tests Debian packages, as well as the proposed Boost.Process library and the Boost.Thread). Multithreading and Multiprocessing in the compiled kernel seems to be functioning fine, but multitasking has a couple of intermittent issues when used in user space code (see below). The kernel -IS- doing a concurrent boot in runlevel 2, though. * NUMA, swap, mm, slab allocator - Memory seems to be fine. I've compiled both LLVM + Clang and Linux multiple times on the clang built kernel, and I threw some "eat your memoryez" programs from the Debian repositories at it; doesn't look like there's trouble here. * Network stack (IPv4) - The IPv4 stack is fine, except for IPSec (see the notes on crypto support below). Netfilter and IPv6 both (separately) caused ICEs when I initially tried to build the network stack. I think I fixed this issue elsewhere (getting harddrive drivers to compile), so this may not be in bad shape at all (harddrive issue was an ICE caused by some seemingly legal conditional operators semantics in the kernel. The problem is specifically in CodeGen, which is the part of the clang that I've hacked the most, so it's likely this is a bug I introduced on my local fork). * Drivers and Firmware Stuff - Drivers have all been relatively well behaved thus far. However, I've had to be very picky, as any driver that depends on crypto stuff (even the basic kernel crc routines) breaks things. Mostly, I've just compiled what my laptop has needed and the generic drivers needed for my microATX desktop box/running the kernel in Qemu. I have added some of less obscure drivers to the build configuration, though. Here's a short list of things that work on my laptop (all the open source drivers are clang-compiled): * Graphics and sound. I installed Flash and listened to some BoostCon videos from last year to confirm this. * Keyboard. No backlight, though, but I never had that working in the first place. * DVD/CDROM. * Touchpad. * Various USB stuff. * iSight. * Speaker Non-Essential Subsystems Failing at Compile Time ================================================ Again, I'm possibly forgetting things here. I should also note that it's very possible that my hacks to either clang, Linux or the Linux build setup caused this. * SELinux, Posix ACLs, IPSec, eCrypt, anything that uses the crypto API - None of these will compile, due to either an ICE or variable-length arrays in structures (don't remember which, it's in my notes somewhere). If it's variable-length arrays or another intentionally unsupported GNUtension, I'm hoping it's just used in some isolated implementation detail (or details), and not a fundamental part of the crypto API (honestly just haven't had a chance to dive into the crypto source yet). I'm really hoping it's an issue in Clang, though, as it's easier for me to hack Clang and I'm trying to avoid kernel patches as much as possible. * IPv6 and Netfilters/Router stuff - Some of this is tied to the above issues with the crypto API, but IPv6 and Netfilters each have their own fatal errors. * Virtualization - Virtualization is the only thing I haven't done any work on yet. I tried compiling minimal Xen support, ran into a fatal error, and put it on my "Get the basics functional first". Fatal Subsystem Failures Temporarily Averted by Compiling with GCC ================================================================== These two issues were already known when I started working on this, and conveniently, people had patched the Linux build system to build these systems with GCC, but were still unable to build a usable kernel. As I'm lazy, I took this work and reused it. * VDSO - VDSO breaks in strange ways with clang, at least, it did a week ago when I put some time into investigating this. ATM, building VDSO with GCC works, but I believe that this is still causing issues. I think the issue here is similiar in nature to the issue with LKMs. * Boot - The very early kernel boot code breaks with clang, because of obscure inline assembly GNUtensions (.code16gcc stuff). I have no clue what needs to be done to fix this, but as I actually know where this problem is, it should be (relatively) easy to fix. Crippling Problems ================== * Modules - Module loading is totally broken. I'm pretty sure I just figured out why, though (I'd elaborate, but this is pretty lengthy, and I might as well just go implement it). Brief closing notes - I'm adapting a slightly aged POSIX test suite developed by some Intel hackers for use in stress testing/functionality testing/unit testing clang compiled kernels. This is something of a side project for me (I have school and what not), but my intention is to maintain a Linux git repo with the source patches/build set up needed to compile with a matching Clang git repo. Hopefully in a few weeks/couple of months, with intensive automated testing and fixes to the above issues, Clang + Linux will be feasible for producing production quality Linux kernels. I'm going to try to clean up some of my modifications to Clang (mostly hacks in CodeGen stuff, local labels (not 100% done yet), explicit register variables, a more complete implementation of GNU inline assembly constraints). Some of my changes implement the sort of cryptic GNUtensions that I sense most Clang devs would find distasteful (I haven't added support for anything explicitly stated as unsupported on the clang website). P.S. boot logs: http://gist.github.com/644488http://gist.github.com/644490 I saved build times somewhere. Depending on how the configuration is tweaked/the SMP support of the kernel that I'm building on top of, Clang builds Linux in about 13-15 minutes. Back to hacking :) - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkzFI6IACgkQO/fqqIuE2t7RRQCaAzpFjAC7T9qTHunR69NNZzfk Sj8AnR28NvtaRQv0fqM8S8gA+1UieqLj =3ZwN -----END PGP SIGNATURE-----

Jul	SEP	Oct
	07
2014	2015	2016