Discussion:
packet socket support
(too old to reply)
s***@public.gmane.org
2014-04-02 15:33:02 UTC
Permalink
Hi, Lazlo,

I don't follow all of the details of your technical issues. But I have thought about implementing raw sockets in the past. I recall that the biggest issue is routing of incoming packets to raw socket listeners.

I tend to agree with your analysis. Basically, the solution would be to replace almost all of the existing under net/uip/* (which makes since because that does only IP) and change a lot of net/* as well. I have a few thoughts, but I think you are further along in the design that I every was.

Greg
d***@public.gmane.org
2014-04-02 14:26:46 UTC
Permalink
Hi everyone!

I'm working on a patch that will allow using packet sockets in nuttx.

I have written a tiny example application that calls the respective socket functions and I used gdb to step into the calls made to determine where I need to modify something in order to make sockets with domain=PF_PACKET, type=SOCK_RAW work.

Based on the information from gdb I patched psocket_socket() and psock_bind() so they at least do not complain but do nothing intelligent but return success.

I left socket() unmodified, so it will still allocate a internal socket structure for the packet socket. But if and what kind of connection should be allocated by psock_bind() i am unsure of. man 7 packet tells me, bind for AF_PACKET will only used sockaddr_ll.sll_protocol and sockaddr_ll.sll_ifindex.

From working with packet sockets on linux I have experienced that one needs to bind() a packet socket, even if one is only sending, otherwise write() will fail with "no such device".

What I also did is copy the netpacket/packet.h from linux to have the struct sockaddr_ll defined that is used at least with bind().

From my understanding I will also need to make recvfrom() and sendto() somehow read from or write to the buffer of the corresponding device from g_netdevices. This is the point where I'm not sure how to proceed.

I know for sendto() that I need to lookup the correspondig uip_driver_s instance in g_netdevices using the information from sockaddr_ll.ifindex but how do I get the sockaddr_ll struct that was passed to bind() when I'm inside sendto()? There needs to be a corresponding connection (allocated by something like uip_llsalloc()) attached to the *psock that links to the respective driver I guess.

For recvfrom() I know that I need to copy all packets with respect to sockaddr_ll.sll_protocol before they are passed to uIP.

All this makes me think I will also need to adjust parts of uIP and add a uip_llsconn.c even tough packet sockets should bypass that stack and have no connection.

I'd really love to contribute but feel a bit stuck rightnow.

Let me know if I should post my changes.

Lazlo
d***@public.gmane.org
2014-04-07 14:33:15 UTC
Permalink
Hi Greg,

I have made some progress in implementing packet socket support for nuttx/uip and have a question about the semaphore in the recvfrom_s struct and the uip_lockedwait() function.

I have written a pkt_recvfrom() that does the following:
- call uip_lock()
- call recvfrom_init()
- assign state.rf_cb = uip_pktcallbackalloc()
- assign the cb attributes (flags, priv and event=recvfrom_pktinterrupt())
- calls uip_lockedwait(&state.rf_sem);
- call uip_pktcallbackfree()
- call recvfrom_result()
- call uip_unlock()
- call uip_recvfrom_uninit()

The problem I have is that the uip_lockedwait() never returns / the semaphore from the recvfrom_s struct (state.rf_sem) is not released (no call to sem_post()?).

As the original comment above the call to uip_lockedwait() in recvfrom.c explains the uip_lockedwait() is exited when either reception is complete or a error/timeout occures.

My question is, what must i call on rf_sem (sem_give()?) to release the sem. or even better, where the code resides that already does this for any other protocol so i can extrapolate.

Is my assumption correct that the sem is/should be released by some part of recvfrom.c (maybe recvfrom_pktinterrupt() or recvfrom_newpktdata()?)

Lazlo
Daniel Laszlo Sitzer
2014-04-07 14:57:25 UTC
Permalink
Urg ... I think I found the spot.

It was just at the end of recvfrom_pktinterrupt(). I added a
sem_post(&pstate->rf_sem) there and now it works as in, recv() retuns the
number of received bytes and my test applications continues reading the
next frames recevied : )


On Mon, Apr 7, 2014 at 4:33 PM, <dlsitzer-***@public.gmane.org> wrote:

>
>
> Hi Greg,
>
> I have made some progress in implementing packet socket support for
> nuttx/uip and have a question about the semaphore in the recvfrom_s struct
> and the uip_lockedwait() function.
>
> I have written a pkt_recvfrom() that does the following:
> - call uip_lock()
> - call recvfrom_init()
> - assign state.rf_cb = uip_pktcallbackalloc()
> - assign the cb attributes (flags, priv and event=recvfrom_pktinterrupt())
> - calls uip_lockedwait(&state.rf_sem);
> - call uip_pktcallbackfree()
> - call recvfrom_result()
> - call uip_unlock()
> - call uip_recvfrom_uninit()
>
> The problem I have is that the uip_lockedwait() never returns / the
> semaphore from the recvfrom_s struct (state.rf_sem) is not released (no
> call to sem_post()?).
>
> As the original comment above the call to uip_lockedwait() in recvfrom.c
> explains the uip_lockedwait() is exited when either reception is complete
> or a error/timeout occures.
>
> My question is, what must i call on rf_sem (sem_give()?) to release the
> sem. or even better, where the code resides that already does this for any
> other protocol so i can extrapolate.
>
> Is my assumption correct that the sem is/should be released by some part
> of recvfrom.c (maybe recvfrom_pktinterrupt() or recvfrom_newpktdata()?)
>
> Lazlo
>
>
Gregory Nutt
2014-04-07 15:55:34 UTC
Permalink
Hi, Lazlo,


> Urg ... I think I found the spot.

I am glad that you found the problem.  I could not think of anything to say to help you B^)

Are you going to contribute these changes back to NuttX.  SOCK_RAW would be a welcome addition.

If so, they you probably should start submitting some bits and pieces for review.  I will review both the design and coding style and get the code into the GIT repository.

Greg
Daniel Laszlo Sitzer
2014-04-08 11:03:11 UTC
Permalink
Hello Greg,

If you like I will prepare a patch with the things I modified in order
receive using a packet socket and send it to you via mail. This might take
a bit since I have not created a feature branch and will need to diff
against the sourceforge repo and I'am still working on the transmission
part.

I'll keep you updated.



On Mon, Apr 7, 2014 at 5:55 PM, Gregory Nutt <spudarnia-/***@public.gmane.org> wrote:

>
>
> Hi, Lazlo,
>
> > Urg ... I think I found the spot.
>
> I am glad that you found the problem. I could not think of anything to
> say to help you B^)
>
> Are you going to contribute these changes back to NuttX. SOCK_RAW would
> be a welcome addition.
>
> If so, they you probably should start submitting some bits and pieces for
> review. I will review both the design and coding style and get the code
> into the GIT repository.
>
> Greg
>
>
>
Gregory Nutt
2014-04-08 14:32:30 UTC
Permalink
Hi, Lazlo,

> If you like I will prepare a patch with the things I modified in order receive using a packet socket and send it to you via mail. This might take a bit since I have not created a feature branch and will need to diff against the sourceforge repo and I'am still working on the transmission part.


You should do what you feel is best and what you are most comfortable with.  I could possible help you if I could see the code, but that is not essential as long as you are making good progress.  Please do keep me updated.

Greg
dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-12 15:32:46 UTC
Permalink
Hi Greg & list


I finally managed to create a patch that contains my changes for the AF_PACKET support.


The reason why it took me so long was that I was assigned another task while working on the packet support and just got back to where I left off. In addition to that, I had to restructure my changes to be on their own branch so I would be able to create patches simply by calling format-patch ... which took me a day of git cherry-picking.


After the patch is applied one needs to set CONFIG_NET_PKT=y and CONFIG_EXAMPLE_NETPKT=y (target side test application)


What I noticed is, even though the line 'source "$APPSDIR/examples/netpkt/Kconfig"' for apps/examples/Kconfig is part of my patch, it might be necessary to apply this change manually, otherwise the netpkt test application will not be visible in the menuconfig.


When the image is build and uploaded (with nsh as main application), i usually run "netpkt -a" to test reception and transmission.


There is also another host-side tool for testing which is not yet included in the patch, but I plan on moving its code also into apps/examples/netpkt and have it build for the host while the nuttx build for the target is executing (like the nettest example does).


Maybe you/someone could have a look at the general approach of how I integrated the packet socket support into the net and uip subsystems and let me know if there are some obvious or not so obvious issues.


Some general notes:
- The patch modifies the stm32_eth.c driver, but would work with any other ethmac driver I think.
- The code for transmission is only implemented in net_send_unbuffered.c but not in net_send_buffered.c


There are a couple of issues with the packet socket support which I have trouble fixing:


One thing is that when I write data into a AF_PACKET socket, it is not transmitted. At least in my current version, but I think I had it working already.
Reception seems to work as far as I can tell.


And there is one more thing: After I run my netpkt test application all networking functionality no longer works : ( (uip_lockedwait() then never returns)


What I have noticed is that after calling my netpkt test application, there is still a active callback arround / uip_pktcallback() gets called, when I'd expect it to be gone.


Anyway, I hope I will be able to fix these issues so this could be part of the nuttx networking subsystem.


Lazlo
spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-12 16:00:39 UTC
Permalink
Hi, Lazlo,

Very cool! This is a nice contribution.

I see that one of the header files (netpack/netpacket.h) is LGPL. You only need a few common interfacing definitions from this file so I think I can legitmately create a BSD compatible version of that header file.

I will let you know when I have everything available in the repository.

Thanks again,
Greg
spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-12 19:10:23 UTC
Permalink
Hi, Lazlo,

Everything is checked in and should be available in the repository.

Nice job! I have not test anything but the design looks like a good NuttX drop-in.

I did make a few changes for coding style and I tried to better detangle some the AF_PACKET, SOCK_RAW, and socketaddr_ll references when CONFIG_NET_PKT is disabled. Those should be harmless.

I stripped out most everything from the netpacket.h and changed types so that the only commonality between the NuttX version of packet.h and the LGPL version is the name of the structure. So it is certainly legitimate to now git it a BSD license.

Everything looks good! Thank you for the contribution. I hope that people can make some good use of your work.

Greg
Daniel Laszlo Sitzer dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-13 09:50:52 UTC
Permalink
Hi Greg!

Thank you so much for your help!

I already found one commit I missed putting in my patch but that will not
be the last change I think.
I hope I will be able to make the feature more stable in the next couple of
days and keep you updated.

Lazlo


On Thu, Jun 12, 2014 at 9:10 PM, spudarnia-/***@public.gmane.org [nuttx] <
nuttx-***@public.gmane.org> wrote:

>
>
> Hi, Lazlo,
>
> Everything is checked in and should be available in the repository.
>
> Nice job! I have not test anything but the design looks like a good NuttX
> drop-in.
>
> I did make a few changes for coding style and I tried to better detangle
> some the AF_PACKET, SOCK_RAW, and socketaddr_ll references when
> CONFIG_NET_PKT is disabled. Those should be harmless.
>
> I stripped out most everything from the netpacket.h and changed types so
> that the only commonality between the NuttX version of packet.h and the
> LGPL version is the name of the structure. So it is certainly legitimate
> to now git it a BSD license.
>
> Everything looks good! Thank you for the contribution. I hope that
> people can make some good use of your work.
>
> Greg
>
>
dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-18 15:32:05 UTC
Permalink
Hi Greg!


With some help I managed to pin down and fix a couple of problems with the packet socket support.


One is that I forgot to properly free the socket in psock_close() and hence all calls after the first to my test application caused uip_lockedwait() to never return.


Another problem, closely related, was to have socket() return failure when there is no free socket/connection to be allocated.


Also missing was the spot where the number of bytes to transmit is set in the uip_driver_s instance in net_send_unbuffered.c and hence, stm32_transmit() was never called.


And I also created a patch for the commit I forgot to add last time, that will notify the ethmac using dev->txavail() that something is to be transmitted.


Lazlo
Gregory Nutt spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-18 17:10:01 UTC
Permalink
Hi, Lazlo,

Okay. Everything is checked in and available in the repository. Thanks
for the updates.

A heads up: I am slowly over time trying to better integrate and
modularize the networking code. That code started as a uIP "bag" on the
side of NuttX but has gone very high coupled over time. A first step to
do this is the partition files into "modules" and to keep these in
separate directories. If you look in net/ you will see that I have
started that some time back. Today I also moved the TCP, UDP, and
packet socket logic into net/tcp/, net/udp, and net/pkt directories.
Let me know if you see any problems.

Next steps will be to detangle the header files, make sure that the
interfaces are well defined, and to clean up naming conventions. But
there is still long wayto go and I am not in a rush to complete that.
Just getting all files into appropriate directories will be a good
start. I still envision the directories, net/socket and net/netdev.

Greg

On 6/18/2014 9:32 AM, dlsitzer-***@public.gmane.org [nuttx] wrote:
> Hi Greg!
>
> With some help I managed to pin down and fix a couple of problems with
> the packet socket support.
>
> One is that I forgot to properly free the socket in psock_close() and
> hence all calls after the first to my test application caused
> uip_lockedwait() to never return.
>
> Another problem, closely related, was to have socket() return failure
> when there is no free socket/connection to be allocated.
>
> Also missing was the spot where the number of bytes to transmit is set
> in the uip_driver_s instance in net_send_unbuffered.c and hence,
> stm32_transmit() was never called.
>
> And I also created a patch for the commit I forgot to add last time,
> that will notify the ethmac using dev->txavail() that something is to
> be transmitted.
>
> Lazlo
>
>
Daniel Laszlo Sitzer dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-19 09:04:05 UTC
Permalink
Hi Greg,

Thank you for integrating my patch!

I already noticed that you and Alan are working on 802.11 and inferred from
that that you guys will also rearrange the networking subsystem. I don't
see any serious problem with that, just the opposite. I'm looking forward
to your changes.

I also don't plan on changing to much of my patch. Yesterday I started to
add just a few lines to allow to skip sending ARP requests when a packet
socket is used. I added a d_noarp to the uip_driver_s that is set 1 in
pktsend() and cleared by arp_out(), which will simply reset d_noarp=0 and
return.

I'm not 100% happy with that solution. I have thought about putting the
d_noarp information into the connection structure, so it will not disturb
other sockets. The problem is that I did not manage to access the
connection from within arp_out().

Besides that, I will add the host-side testing tool in my next patch, so
others can find and use it in the netpkt/ directory and give the patch some
more testing.

Lazlo



On Wed, Jun 18, 2014 at 7:10 PM, Gregory Nutt spudarnia-/***@public.gmane.org [nuttx] <
nuttx-***@public.gmane.org> wrote:

>
>
> Hi, Lazlo,
>
> Okay. Everything is checked in and available in the repository. Thanks
> for the updates.
>
> A heads up: I am slowly over time trying to better integrate and
> modularize the networking code. That code started as a uIP "bag" on the
> side of NuttX but has gone very high coupled over time. A first step to do
> this is the partition files into "modules" and to keep these in separate
> directories. If you look in net/ you will see that I have started that
> some time back. Today I also moved the TCP, UDP, and packet socket logic
> into net/tcp/, net/udp, and net/pkt directories. Let me know if you see
> any problems.
>
> Next steps will be to detangle the header files, make sure that the
> interfaces are well defined, and to clean up naming conventions. But
> there is still long wayto go and I am not in a rush to complete that. Just
> getting all files into appropriate directories will be a good start. I
> still envision the directories, net/socket and net/netdev.
>
> Greg
>
> On 6/18/2014 9:32 AM, dlsitzer-***@public.gmane.org [nuttx] wrote:
>
>
>
>
>
>
dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-19 10:34:02 UTC
Permalink
Hi Greg,

I found one tiny problem. When you moved/renamed the packet socket sources, the pkt/Make.defs still contained to old source file names.

Lazlo
spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-19 13:35:06 UTC
Permalink
Hi, Lazlo,

> I found one tiny problem. When you moved/renamed the packet socket sources, the pkt/Make.defs still contained to old source file names.

Thanks! The fix is checked-in.

Sorry about the chaos, but I think that this structural re-organization of the networking code is important for future developments.

Greg
spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 12:54:40 UTC
Permalink
Hi, Lazlo,

I do need to make one change to the packet socket logic. Hopefully just a packaging change with no real design change.

The problem is that unbuffered TCP sends and raw packet sends are implemented in the same file. But buffered TCP sends are implement in a different file. That means that one can enable packet sockets and unbuffered TCP sockets, but you can't enabled packet sockets and buffered TCP sockets.

The solution is simple: Divide the code into three files pkt_send, tcp_buffered_send, and tcp_unbuffered_send. That will require duplication of a small amount of logic, but I think will not be too bad.

Greg
Daniel Laszlo Sitzer dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 13:34:21 UTC
Permalink
Hi Greg,

from my side, I think there should be no problem with that change.

I myself was unhappy about the fact that I only placed the code in
net_send_unbuffered.c, so net_send_buffered.c does not contain the feature.
I just did not want to get into the complexity of also buffering the raw
frames, written to the AF_PACKET socket. And at that time it just didn't
came to my mind that I could simply put my code into an extra file ; (

Besides, the changes I'm currently working on will only be minor (I hope),
like making uIP only send ARP requests for packet sockets if the frame
contains an IP packet (got that idea from a colleague). This feature does
not work yet but I have a version that will skip ARP for packet sockets
altogether (which finally makes my change do something useful, not just
send arp packets when you write a something into the packet socket like
now).

In fact, that change concerns netpkt_send() in net_send_unbuffered.c but I
think I will manage moving my change to the new file you suggested.

Lazlo



On Tue, Jun 24, 2014 at 2:54 PM, spudarnia-/***@public.gmane.org [nuttx] <
nuttx-***@public.gmane.org> wrote:

>
>
> Hi, Lazlo,
>
> I do need to make one change to the packet socket logic. Hopefully just a
> packaging change with no real design change.
>
> The problem is that unbuffered TCP sends and raw packet sends are
> implemented in the same file. But buffered TCP sends are implement in a
> different file. That means that one can enable packet sockets and
> unbuffered TCP sockets, but you can't enabled packet sockets and buffered
> TCP sockets.
>
> The solution is simple: Divide the code into three files pkt_send,
> tcp_buffered_send, and tcp_unbuffered_send. That will require duplication
> of a small amount of logic, but I think will not be too bad.
>
> Greg
>
>
spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 14:01:09 UTC
Permalink
Hi, Lazlo,

Buffering output on packet sockets is not really a must-have feature. It might improve application performance, since applications would not have to wait for the send to complete. But, for network performance, I don't expect it would have very much effect since there is only a little difference between explicit queuing of buffers and implicit queuing by make the application wait.

For TCP, write buffering is very important, however. Mostly because of the delayed ACK on most networks. You must be able to write ahead of the ACKs in order order to get full network performance and you cannot do that unless you queue output data. The network performance lossis due the delayed ACK timeouts: Essentially every TCP write has to stall until the delayed ACK timeout occurs and the data finally gets ACKed.

So I don't see any pressing need to change anything with the packet sockets unless you want to have your application that needs to exploit the time while the packet is actually being sent.

On the other hand, the buffered output for packet sockets would be relatively simple because you do not have to deal with the TCP sequence numbers and ACKing.

Greg
spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 14:02:03 UTC
Permalink
Ooops, I did not read carefully...

> In fact, that change concerns netpkt_send() in net_send_unbuffered.c but I think I will manage moving my change to the new file you suggested.

I have already made this change. I am verifying all build configurations and will check the code into the netio branch in the next few minutes. No need for you do anything except resync.

Greg
spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 14:07:26 UTC
Permalink
Here is the change: http://sourceforge.net/p/nuttx/git/ci/827c7af539993a89f996e29a6836e911f9cc409b/ http://sourceforge.net/p/nuttx/git/ci/827c7af539993a89f996e29a6836e911f9cc409b/
dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 14:02:53 UTC
Permalink
Hi Greg,

me one more time. I just created a patch (3 parts) that basically set the IFF_NOARP flag when something is transmitted using a packet socket and that bit is checked by arp_out() which simply returns if the flag is set.

I thought it might be easier if I submit my change now before you move net netpkt_send(). If its not possible, thats also okay, then I'll adjust the code after your change.

The code that will make arp only send requests when the frame contains an IP packet is still work in progress.

PS: Sorry for this clumsy patch. you can just throw away the unrelated comments .. I didn't mange to have my patch without them.

Lazlo
spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 14:09:04 UTC
Permalink
Oops, just moved the changes to master. I will incorporate your changes, no need for you to do anything except to verify that I did not make any mistakes.

Greg
Gregory Nutt spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 14:36:15 UTC
Permalink
All of you changes should be available in the GIT master branch now.
Please try them out.

Greg

On 6/24/2014 8:09 AM, spudarnia-/***@public.gmane.org [nuttx] wrote:
>
> Oops, just moved the changes to master. I will incorporate your
> changes, no need for you to do anything except to verify that I did
> not make any mistakes.
>
> Greg
>
>
Daniel Laszlo Sitzer dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 14:43:30 UTC
Permalink
Hi Greg,

I just pulled the latest changes and tried building. It fails in
net_sockets.c:50 because CONFIG_IOB_BUFSIZE in "uint8_t
io_data[CONFIG_IOB_BUFSIZE];" is undeclared, while my CONFIG_NET_IOB is not
set.

Besides that I'll keep checking what happened in the last commits of yours
and get myself familiar with the new structure.



On Tue, Jun 24, 2014 at 4:36 PM, Gregory Nutt spudarnia-/***@public.gmane.org [nuttx] <
nuttx-***@public.gmane.org> wrote:

>
>
> All of you changes should be available in the GIT master branch now.
> Please try them out.
>
> Greg
>
>
> On 6/24/2014 8:09 AM, spudarnia-/***@public.gmane.org [nuttx] wrote:
>
>
>
> Oops, just moved the changes to master. I will incorporate your changes,
> no need for you to do anything except to verify that I did not make any
> mistakes.
>
> Greg
>
>
>
>
Gregory Nutt spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 14:56:20 UTC
Permalink
> I just pulled the latest changes and tried building. It fails in
> net_sockets.c:50 because CONFIG_IOB_BUFSIZE in "uint8_t
> io_data[CONFIG_IOB_BUFSIZE];" is undeclared, while my CONFIG_NET_IOB
> is not set.
>
Something needs to be conditionally compiled on CONFIG_NET_IOB. I will
make that change too.

Greg
Gregory Nutt spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 15:08:23 UTC
Permalink
Hi, Lazlo
>> I just pulled the latest changes and tried building. It fails in
>> net_sockets.c:50 because CONFIG_IOB_BUFSIZE in "uint8_t
>> io_data[CONFIG_IOB_BUFSIZE];" is undeclared, while my CONFIG_NET_IOB
>> is not set.
>>
> Something needs to be conditionally compiled on CONFIG_NET_IOB. I
> will make that change too.

I think that the contents of include/nuttx/net/iob.h just need to be
conditioned out when CONFIG_NET_IOB is not set. That change is checked
in but I am still building with CONFIG_NET_IOB=y. Please try again,

Greg
Daniel Laszlo Sitzer dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 15:17:17 UTC
Permalink
Looks better now.

In the meantime I build with NET_IOB=y which successfully compiled and
worked (when I send something via pkt socket, the result is an arp request,
which is what I'd expect without my noarp patch).

Now I pulled the latest code where the noarp is included but now there
seems to be an issue in the late stages of the build.
Make complains that *** No rule to make target ...
include/nuttx/net/uip/uipopt.h', needed by 'os_start.o'

I'm not sure If the problem is with my .config or a Make.defs or Make.dep
file. I will try again, running distclean before to remove the Make.deps ..
.maybe that will help.


On Tue, Jun 24, 2014 at 5:08 PM, Gregory Nutt spudarnia-/***@public.gmane.org [nuttx] <
nuttx-***@public.gmane.org> wrote:

>
>
> Hi, Lazlo
>
>
>
> I just pulled the latest changes and tried building. It fails in
> net_sockets.c:50 because CONFIG_IOB_BUFSIZE in "uint8_t
> io_data[CONFIG_IOB_BUFSIZE];" is undeclared, while my CONFIG_NET_IOB is not
> set.
>
> Something needs to be conditionally compiled on CONFIG_NET_IOB. I
> will make that change too.
>
>
> I think that the contents of include/nuttx/net/iob.h just need to be
> conditioned out when CONFIG_NET_IOB is not set. That change is checked in
> but I am still building with CONFIG_NET_IOB=y. Please try again,
>
> Greg
>
>
>
Gregory Nutt spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 15:25:08 UTC
Permalink
Hi, Lazlo,
>
> In the meantime I build with NET_IOB=y which successfully compiled and
> worked (when I send something via pkt socket, the result is an arp
> request, which is what I'd expect without my noarp patch).
>
> Now I pulled the latest code where the noarp is included but now there
> seems to be an issue in the late stages of the build.
> Make complains that *** No rule to make target ...
> include/nuttx/net/uip/uipopt.h', needed by 'os_start.o'
>
> I'm not sure If the problem is with my .config or a Make.defs or
> Make.dep file. I will try again, running distclean before to remove
> the Make.deps .. .maybe that will help.
>

I am in the process of renaming header files. So
include/nuttx/net/uip/uipopt.h is now include/nuttx/net/netconfig.h.

Are you building on Windows? If so, you might need 'make clean_context'
to get rid of the fake symbol links.

Greg
Daniel Laszlo Sitzer dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 15:37:23 UTC
Permalink
I am building on a linux.

Okay, so I ran make clean_context and distclean as well, setup by board
config again and now it builds successfully (NET_IOB=n, NET_PKT=y).

I have not tested the image on the board yet but will do so in the next
half hour.

The only thing that I'm missing is that somehow the entry in
applications->examples->netpkt is now missing.
This may be due to some of by rebasing ... I'm not sure yet.


On Tue, Jun 24, 2014 at 5:25 PM, Gregory Nutt spudarnia-/***@public.gmane.org [nuttx] <
nuttx-***@public.gmane.org> wrote:

>
>
> Hi, Lazlo,
>
>
> In the meantime I build with NET_IOB=y which successfully compiled and
> worked (when I send something via pkt socket, the result is an arp request,
> which is what I'd expect without my noarp patch).
>
> Now I pulled the latest code where the noarp is included but now there
> seems to be an issue in the late stages of the build.
> Make complains that *** No rule to make target ...
> include/nuttx/net/uip/uipopt.h', needed by 'os_start.o'
>
> I'm not sure If the problem is with my .config or a Make.defs or Make.dep
> file. I will try again, running distclean before to remove the Make.deps ..
> .maybe that will help.
>
>
> I am in the process of renaming header files. So
> include/nuttx/net/uip/uipopt.h is now include/nuttx/net/netconfig.h.
>
> Are you building on Windows? If so, you might need 'make clean_context'
> to get rid of the fake symbol links.
>
> Greg
>
>
>
dlsitzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-24 16:10:01 UTC
Permalink
Everything fine!

My fault. Forgot to enable my own feature ; )

Thank you for your help!

Lazlo
Gregory Nutt spudarnia-/E1597aS9LQAvxtiuMwx3w@public.gmane.org [nuttx]
2014-06-19 13:27:09 UTC
Permalink
Hi, Lazlo,


> I already noticed that you and Alan are working on 802.11 and inferred from that that you guys will also rearrange the networking subsystem. I don't see any serious problem with that, just the opposite. I'm looking forward to your changes.


This is a difficult port.  The BSD code is huge and complex.  It will take some time to get the size and down and it get it fully integrated (I don't even understand the full BSD design yet!).

The only change that I want to do is to incorporate a common buffering scheme.  I have a start at the code now at net/iob.  The IEEE 802.11 code requires this kind of buffering and I think it would be good for TCP as well.  The idea is that instead of allocating a list of large, packet-size buffers, you allocate many tiny buffers and these tiny buffers can be linked to produce an I/O buffer chain.  Then you manage lists of I/O buffer chains.  So this keeps the buffering requirements down to a minimum.

There are then operators on the I/O buffer chain that make it just like working with a flat memory allocation.  I plan to convert the TCP write buffering logic to use I/O buffer chains as soon as I have a free day (which has not happened lately).

I will need to come up with some kind of throttling scheme so that TCP read-ahead does not hog all of the I/O buffers.

> I also don't plan on changing to much of my patch. Yesterday I started to add just a few lines to allow to skip sending ARP requests when a packet socket is used. I added a d_noarp to the uip_driver_s that is set 1 in pktsend() and cleared by arp_out(), which will simply reset d_noarp=0 and return.
>
> I'm not 100% happy with that solution. I have thought about putting the d_noarp information into the connection structure, so it will not disturb other sockets. The problem is that I did not manage to access the connection from within arp_out().

Yes, that would have to be  property of the connection, not of the device.  At the level that ARP works, it would not be simple to get connection information. arp_out is called from the device driver.  So perhaps that indication does have to go into the device structure?

You don't have to add a boolean, you could add another bit to d_flags (saves 1 byte).  That bit would have to be cleared by the device driver or uip_input on every poll (yech) and set only for packet transmissions.

> Besides that, I will add the host-side testing tool in my next patch, so others can find and use it in the netpkt/ directory and give the patch some more testing.

Eventually, we will to get the packet socket support into the buffered send logic as well (currenly it only works with unbuffered send).

Greg
Continue reading on narkive:
Loading...