Discussion:
NXFLAT, ROMFS, and mmap()
Gregory N
2012-03-27 16:10:26 UTC
Permalink
I'm also interested exploring how the NuttX ROMFS compares to a Linux root file system as far as adding executables.
Executables work great with NXFLAT and ROMFS. NXFLAT is described here: http://nuttx.sourceforge.net/NuttXNxFlat.html . ROMFS is a standard read-only file system described here: http://romfs.sourceforge.net/ . ROMFS supports XIP so that you do not have to copy the data to SRAM; rather you can eXecute-In-Place in the ROMFS file system. ROMFS is unique in that way. NXFLAT mmap's the file and ROMFS support XIP memory mapping.

If you don't use ROMFS then you have to use the mmap() emulation. Without an MMU, you can't really support true mmap(), but NuttX has a mmap() emulation that will copy the whole file into SRAM where it can executed. This is kind of a pricey solution for most SRAM-limited MCUs.

The NuttX root file system is always a psuedo-filesystem. It is just the opposite from Linux. With Linux the root file system must always be some physical block device (if only an initrd ram disk). Then once you have mounted the physical root file system, you can mount other file systems -- including Linux psuedo-filesystems like /proc or /sys. With NuttX, the root file system is always a pseudo-filesystem that does not require any underlying block driver or physical device. Then you can mount real filesystem in the pseudo-filesystem.

This arrangement makes life much easier for the tiny embedded world (but also has a few limitations --- like where you can mount file systems).

So in this case here, ROMFS would not be the root file system. But could be mounted at any location under / in the psuedo-filesystem. If you want executables, you could mount a ROMFS file system at /bin for example.
I'm used to simply populating a /bin folder in an embedded root file system image with the programs I want to execute at run time.
If you are using, say, a FAT file system and a the copy-to-SRAM mmap() emulation. Then it could be just like this. However, that is not usually a good solution unless you have tons of RAM to copy images because FAT is not XIP.

ROMFS has can reside in any random-access FLASH and XIP will work if that FLASH supports code execution. The ROMFS image can be built-into the main program on the MCU's internal FLASH using the nuttx/tools/mkromfsetc.sh script. Or the ROMFS image could be written to an external NOR FLASH using standard techniques.

In these cases, I think that the procedure is still more or less as you would have to do it in Linux. You have to create the file system image, run genromfs to create a ROMFS image of the file system, and then install the ROMFS image in FLASH. ROMFS is a read-only file-system so you would never be able to add anything to the bin/ folder, even with NuttX.

A bigger issue is symbol tables. That will probably cause you more problems. When external NXFLAT programs are loaded, they have to be dynamically linked with the base code. This requires two symbol tables: One in the NXFLAT module that enumerates all of the symbols needed by the NXFLAT module (imports). And another in the FLASH base code the enumerates all of the symbols available in the base code (exports).

There are examples of how to do this in apps/examples/nxflat and apps/examples/thttpd. The THTTPD example is particularly cool. There NXFLAT executables are used to implement true CGI, complete with I/O re-direction of standard I/O to the TCP connection.

Greg
avrbasic
2012-03-28 05:55:43 UTC
Permalink
quick question

I assumed that if I add nxflat support and flash write support then

install command would work and work from FLASH without copy to ram?

install writest XIP program to flash, later nsh can execute it?

if this is not possible, then only way is to make romfs WRITEABLE?

like command RESTORE romfs image as whole, not adding files
just replacing all romfs image?

Antti
I'm also interested exploring how the NuttX ROMFS compares to a Linux root file system as far as adding executables.
Gregory N
2012-03-28 13:01:22 UTC
Permalink
Hi, Antti,
Post by avrbasic
I assumed that if I add nxflat support and flash write support then
install command would work and work from FLASH without copy to ram?
install writest XIP program to flash, later nsh can execute it?
Are you talking about the NSH install command at apps/system/install? I have never used that and I don't know very much about it. That logic is undocumented and I don't know enough about it to say very much.

I don't believe that this install command uses any file system to hold the executable and it does not depend on NXFLAT either. I think it just copies a pre-linked, binary executable image from a file (say on an SD card) into MCU FLASH and then creates a little directory structure somewhere but I am not sure. It does have a help command and the code is pretty easily understandable so if you wanted to use it, it should not be too difficult to adapt.

NOTE: apps/system/install is only used by the configs/vsn/nsh configuration and may have other board dependencies. If you have difficulties, you might need to check other logic in configs/vsn/src.

Yes, so Uros' apps/system/install command is yet another way to get an executable off of a file system like FAT and into an executable format. So previously I said that you could copy a file to SRAM for execution (using mmap()). Let's extend that and say that you can also copy a file into the MCU FLASH for execution (using apps/system/install).

Using an external serial FLASH with Uros' install command would be a very low-cost way to get a configurable system.

Hmmm... but isn't the MCU FLASH usually locked on most modern products?
Post by avrbasic
if this is not possible, then only way is to make romfs WRITEABLE?
like command RESTORE romfs image as whole, not adding files
just replacing all romfs image?
ROMFS is inherently read-only. Yes, you can replace the whole image as you say, but you cannot change individual files within the ROMFS image.

It sounds like you are thinking about a NSH command to copy an entire ROMFS image from some media (SD, serial FLASH, ...) into MCU FLASH? Yes, that could be done too using something base on apps/system/install.

Greg
Elias Önal
2012-03-28 13:21:17 UTC
Permalink
I have only worked with the STM32 so far, but for that MCU you can
unlock the flash while in privileged mode. (MSP and not PSP being used)
So I guess someone could add a flash-unlock API to NuttX or even simply
unlock the flash at start-up.
Post by Gregory N
Using an external serial FLASH with Uros' install command would be a
very low-cost way to get a configurable system.
Hmmm... but isn't the MCU FLASH usually locked on most modern products?.
avrbasic
2012-03-28 14:15:02 UTC
Permalink
Hi

OK, understood! yes, I looked the vsn/install
and for romfs, yes REPLACE all at once rewrite

MCUs sure can overwrite own code.. that no big deal
so can sure install into own flash if code drivers are in place

Antti
Post by Gregory N
Hi, Antti,
Post by avrbasic
I assumed that if I add nxflat support and flash write support then
install command would work and work from FLASH without copy to ram?
install writest XIP program to flash, later nsh can execute it?
Are you talking about the NSH install command at apps/system/install? I have never used that and I don't know very much about it. That logic is undocumented and I don't know enough about it to say very much.
I don't believe that this install command uses any file system to hold the executable and it does not depend on NXFLAT either. I think it just copies a pre-linked, binary executable image from a file (say on an SD card) into MCU FLASH and then creates a little directory structure somewhere but I am not sure. It does have a help command and the code is pretty easily understandable so if you wanted to use it, it should not be too difficult to adapt.
NOTE: apps/system/install is only used by the configs/vsn/nsh configuration and may have other board dependencies. If you have difficulties, you might need to check other logic in configs/vsn/src.
Yes, so Uros' apps/system/install command is yet another way to get an executable off of a file system like FAT and into an executable format. So previously I said that you could copy a file to SRAM for execution (using mmap()). Let's extend that and say that you can also copy a file into the MCU FLASH for execution (using apps/system/install).
Using an external serial FLASH with Uros' install command would be a very low-cost way to get a configurable system.
Hmmm... but isn't the MCU FLASH usually locked on most modern products?
Post by avrbasic
if this is not possible, then only way is to make romfs WRITEABLE?
like command RESTORE romfs image as whole, not adding files
just replacing all romfs image?
ROMFS is inherently read-only. Yes, you can replace the whole image as you say, but you cannot change individual files within the ROMFS image.
It sounds like you are thinking about a NSH command to copy an entire ROMFS image from some media (SD, serial FLASH, ...) into MCU FLASH? Yes, that could be done too using something base on apps/system/install.
Greg
Loading...