dcdev mailing list

Ok, I'm totally crammed for time for the next few weeks so I'm going to keep this short.

The DC protocol is not extensible enough. That was a mistake on my part. I was young, and didn't give it enough thought. I'd never done anything like that before - I was 17.

That said, I think it would be ugly to add a bunch of messages that rely on some kind of $Support or $Key hacking, and that clients would easily mess it up.

I would prefer a totally different message encoding. That was of a regular format. And the hub+client would agree at connection time if they would use the old encoding method or the new one. This way it would be backwards compatible. All hubs would support both methods.

The message encoding might look something like
<command>:<argument>:<argument>:<argument>
where <argument> was \ escaped to protect from internal | or :'s and command was defined to never contain : or |. This way, you could always read a block of bytes into a list/array of arguments with the same decoding rules.

Further more, I think that it should be UTF8 rather than Windows encoded text.

Also, no hub should only provide the new encoding, and there must be a one to one mapping of all the basic messages. So, client developers would be able to support it easily. For example, I should NOT have to edit the code that acts on or generates a message, but should have to edit the code that encodes/decodes messages.

So, I don't want a new 'protocol' but I do want a new encoding scheme, and I would like to add new messages, but all old messages should retain their meaning.

Requirements would be
    - EVERY hub that supports the new encoding scheme MUST support
      the old scheme
    - Every message is encoded/decoded with the same set of rules
    - No binary values
    - One-to-one mapping from old messages to new messages
    - One protocol addition. A new message 'Supports' that identifies
      the commands a client supports sent immediately after it is decided
      that the new encoding method is being used.

So in the end, I would really prefer a new encoding scheme with a defined way of extending it (supports).

But if you want to make a whole new protocol, I think you need to make a whole set of applications, and call them something besides 'Direct Connect'

One last, requirement. Don't use a name like keywords like JONSEXTENTION or NMDC2EX or BCDCNewProto or anything, keep it client agnostic. :p

Thanks Guys -
Jon Hess

The message encoding might look something like
<command>:<argument>:<argument>:<argument>
where <argument> was \ escaped to protect from internal | or :'s and
command was defined to never contain : or |. This way, you could always
read a block of bytes into a list/array of arguments with the same decoding
rules.

why not switch to a 100% text based client (with CR as command separator), someone mentions it earlier. And in any case, why not using XML based protocol, it is fairly easy to handle, well defined (without incorrect interpretation error), easy to extend and provides native UTF8 encoding.

that acts on or generates a message, but should have to edit the code that
encodes/decodes messages.

Requirements would be
     - EVERY hub that supports the new encoding scheme MUST support
       the old scheme
     - Every message is encoded/decoded with the same set of rules
     - No binary values

you have a problem here. You say you only want to modify the function that encodes/decodes message but this implies a generic function and a generic function will not convert the $MyInfo binary value into a non binary one.

     - One-to-one mapping from old messages to new messages
     - One protocol addition. A new message 'Supports' that identifies
       the commands a client supports sent immediately after it is decided
       that the new encoding method is being used.

So in the end, I would really prefer a new encoding scheme with a defined
way of extending it (supports).

IMHO, it is definitely the first thing to do.

Eric
DCTC/dchub

XML is weak in that it would probably add at least 10% if not more to the
size of the message. Hubs are limited by bandwidth, and we do not need
something so generic.

I agree, hub has a limited bandwidth but it cannot be worse than sending a $GetINFO for each user of a hub :)

Eric
DCTC/dchub
PS: your previous mails were only posted to me, not to the mailing list.

> XML is weak in that it would probably add at least 10% if not more to

the

> size of the message. Hubs are limited by bandwidth, and we do not need
> something so generic.

I agree, hub has a limited bandwidth but it cannot be worse than sending a
$GetINFO for each user of a hub :)

And we can compress the stream before sending with librairies like lzo.

Yves
dchub developer

At 08:29 AM 10/27/2003 +0100, you wrote:

why not switch to a 100% text based client (with CR as command separator), someone mentions it earlier. And in any case, why not using XML based protocol, it is fairly easy to handle, well defined (without incorrect interpretation error), easy to extend and provides native UTF8 encoding.

that acts on or generates a message, but should have to edit the code that
encodes/decodes messages.

Requirements would be
     - EVERY hub that supports the new encoding scheme MUST support
       the old scheme
     - Every message is encoded/decoded with the same set of rules
     - No binary values

you have a problem here. You say you only want to modify the function that encodes/decodes message but this implies a generic function and a generic function will not convert the $MyInfo binary value into a non binary one.

     - One-to-one mapping from old messages to new messages
     - One protocol addition. A new message 'Supports' that identifies
       the commands a client supports sent immediately after it is decided
       that the new encoding method is being used.

I wholeheartedly agree with the idea of replacing DC's encoding with a more extensible sort.

However, some issues with the specific suggestions presented:

I see no reason to mandate that new hubs support both the new and old encodings, and certainly not indefinitely. DC client turnover is quick enough that I'd be comfortable in a few months with dropping support for the old encoding, for example, in any hubs I'm involved with.

CR should not be a separator; I find multiline message support valuable, for example, for ensuring pasted source code is nicely formatted.

XML, of course, avoids the problem with the proposed CR-delimited message format, and I find it preferable. Further, the suggestion to use lzo (or even zlib, depending on overhead; I know lzo has significantly less, but zlib's improved compression may be worth it. Something to consider), renders XML quite bandwidth-friendly. It has the additional benefit of encoding commonly used nicks and command and such concisely after it's built up a dictionary, beyond just the XML tags.

However, I'd prefer an a binary encoding to either of those, as a means of resolving the currently extant divide between a socket mode for transferring files and one for transferring commands. This may be an aesthetic point, but I find that quite unelegant. If one is going to homogenize the rest of the protocol encoding, leaving out the file transfers themselves strikes me, at least, as a conspicuous omission which should be justified. Mandating purely textual content, UTF-8 or not, would prevent such a unification of the protocol, and as such should be avoided. By contrast, a protocol designed to natively handle binary data would be quite capable of dealing with file transfer data itself in a common framework as the commands itself, and is thus an improvement over a text-basis for a P2P file transfer protocol.

My broader objection is the insistance on maintaining a one-to-one mapping between commands in the current encoding and commands in the newer encoding. Rather, I view this as an opportunity at least to remove useless commands (which are by my count broadcast $Hello, $GetInfo, $Key, $Lock, $Version, $GetListLen, $MultiConnectToMe, and $MultiSearch at minimum. I'm sure many of you disagree with various listed items, but there has to be a subset nearly all on the list would be willing to remove).

Finally, DC has many semantic, rather than syntactic, limitations, and whilst replacing the syntax certainly aids towards fixing the broken semantics by creating a more extensible environment in which to introduce improved semantics, I feel that one should take the opportunity created by breaking compatibility to introduce functionality such as: standard GetZBlock-type download support, hash support, search ID support, and various other flaws described at http://murdoc.extatic.org/dctng.html.

-cologic

I see no reason to mandate that new hubs support both the new and old
encodings, and certainly not indefinitely. DC client turnover is quick
enough that I'd be comfortable in a few months with dropping support for
the old encoding, for example, in any hubs I'm involved with.

I think even few weeks is reasonnable. Moreover, IMHO, this is mainly a hub problem, client should only use one protocol version, if someone wants to use both protocol, he justs as to install an old version of its client and the new one.

CR should not be a separator; I find multiline message support valuable,
for example, for ensuring pasted source code is nicely formatted.

sure.

XML, of course, avoids the problem with the proposed CR-delimited message
format, and I find it preferable. Further, the suggestion to use lzo (or
even zlib, depending on overhead; I know lzo has significantly less, but
zlib's improved compression may be worth it. Something to consider),
renders XML quite bandwidth-friendly. It has the additional benefit of
encoding commonly used nicks and command and such concisely after it's
built up a dictionary, beyond just the XML tags.

However, I'd prefer an a binary encoding to either of those, as a means of
resolving the currently extant divide between a socket mode for
transferring files and one for transferring commands. This may be an
aesthetic point, but I find that quite unelegant. If one is going to
homogenize the rest of the protocol encoding, leaving out the file
transfers themselves strikes me, at least, as a conspicuous omission which
should be justified. Mandating purely textual content, UTF-8 or not, would
prevent such a unification of the protocol, and as such should be avoided.
By contrast, a protocol designed to natively handle binary data would be
quite capable of dealing with file transfer data itself in a common
framework as the commands itself, and is thus an improvement over a
text-basis for a P2P file transfer protocol.

I totally agree with you, a binary protocol is better, data can be parsed/processed faster and using "chunked" commands, a client/hub can quickly discard packets it does not need/understand.

Finally, DC has many semantic, rather than syntactic, limitations, and
whilst replacing the syntax certainly aids towards fixing the broken
semantics by creating a more extensible environment in which to introduce
improved semantics, I feel that one should take the opportunity created by
breaking compatibility to introduce functionality such as: standard
GetZBlock-type download support, hash support, search ID support, and
various other flaws described at http://murdoc.extatic.org/dctng.html.

I agree at 98% with the content of this page (there is few thing that cannot be applicable (like maxu/maxd in Info)). I also think each task should have an ID (a task is a more of less big set of commands/replies between 2 entities (hub or client)), this will allow a correct handling of error codes. This is discribed in the page for Search/SR handling but it should be extended to all tasks (like the task 0: the login stage) [Note: don't think I found this in a magic box :) some other protocols use this kind of binary encoding]

Eric
DCTC/dchub

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 28 October 2003 11:32, Jacek Sieka wrote:

Some random thoughts on a new protocol:

1) I like the idea about having 2-3 types of commands, type1 -> hubonly,
type2->a specific client, type3->broadcast. This way, new commands can be
added to clients without upgrading the hubs. Each of these commands would
then have a header for the hub, and the rest of the data for the client
(that the hub can check/parse/deny if needed)

Nice idea, adds alot of flexibility, but we need guidelines for implementing new stuff, and a hub may be configurable to not accept certain commands. I'm not 100% fond of passing through "unknown" commands, which in worst case scenario can break clients (although that would be a client bug/hole).

2) I don't like XML. We're running short on bandwidth, so a more
conservative encoding would suit us better imho. CPU btw is afaik not an
issue (dch++ uses about 4% cpu on an average 1500 user hub on a decent
computer, don't remember exact specs, but it's so low that well...), so
perhaps we could use XML + compression if you're fanatic about it, but imho
it's a ferarri when all we need is a fiat...

I agree, but was mostly thinking from the hub point of view. (Thinking about my good old p200 with 32Mb memory taking a few thousand users today in some tests I've run... :-).

3) I don't like a pure binary protocol for many reasons, ranging for hard
to debug to hard to interpret on different platforms due to different byte
order. The w3c hasn't put out any binary protocols for a reason, and yet,
these are the most successful ones out there serving millions of clients...

Yes, a good example here is the H.323 protocol which is a binary protocol ratified by the ITU (International Telecom Union) for VoIP and videoconferencing. This protocol is as complex that several multi-billion companies don't manage to implement it properly. The best known implementations are from Radvision (used in most commercial videoconferencing equipment today), Microsoft (used in Netmeeting) and OpenH323 (open source).
Debugging it is not fun (I've been playing around with it)!
I expect the SIP protocol will gain alot of market share within a few years since it can do the same thing and is a text protocol.

(...)
6) UTF8 is the encoding of choice for me as well...it should suffice for
now, although we should keep an eye open for 4-byte unicode support once
that becomes popular (don't remember if utf8 can encode that or not)

UTF8 supports all kinds of characters (including 2 and 4 byte unicodes) but use generally less space on western european texts. For asian languages on the other hand, UTF16 is the better choice considering space/bandwidth.

In fact, I started writing on a spec for this, taking many good ideas from
jan vidar's dctng.html freely =).

:)

- -- - -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE/nm9SMLjmoUcyZAoRAqUqAJ9emJFNSUCBnKFXuXu6ASTqMHp7lACePBWc
TaJI0DSP0QJIzE7O3o5VwZA=
=dwjb
-----END PGP SIGNATURE-----

On Tuesday, October 28, 2003, 11:32:19, Jacek Sieka wrote:

4) To maintain some of the binary advantages, I'd suggest going back to the
old FOURCC codes, and have the first letter denote command type. Then BCHA
<from-guid> <text without nick> would be a broadcast chat message, HNIC
<nick> a hub validatenick-type command and CCON <to-guid> <from-guid> be
connecttome.

Why include the from-guid in the messages to hub? Hub should know which GUID
to associate with which connection, and including it in the message only
uses unnecessary bandwidth, and hub would have to do unnecessary checks...

-- 
Jernej Simoncic, ender@ena.si
http://www2.arnes.si/~sopjsimo/
http://deepthought.ena.si/

The fewer functions any device is required to perform, the more
perfectly it can perform those functions.
      -- The Principle Concerning Multifunctional Devices

>Other than that. XML is nice. I suspect it will require a few extra CPU
> cycles
>though.

And lots of bandwidth, let's not forget that bandwidth is the bottleneck
resource.

only if the stream is not compressed and that can be very easy to do using the zlib which provides API to do this.

Eric
DCTC/dchub

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Why include the from-guid in the messages to hub? Hub should know which
GUID to associate with which connection, and including it in the message
only uses unnecessary bandwidth, and hub would have to do unnecessary
checks...

Yes, the hub would have to double check to see if the GUID is correct, but after that it will just pass it along to the recipient(s). Saving a few bytes from each and one participant does not make sense when it has to be broadcasted to all users anyway (hundreds or thousands perhaps).
I'm not sure which is faster, checking the validity of a message and then just pass it through, or recomposing it before sending it along.

The fewer functions any device is required to perform, the more
perfectly it can perform those functions.
       -- The Principle Concerning Multifunctional Devices

Exactly! :)

- -j-

- -- - -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE/nn6eMLjmoUcyZAoRAqy8AJ4kM9xywvtX0Rg/66aKRJzF/bQRjACdF8E3
KGk7UFUsKl4gl2sLr+9E47U=
=2wdj
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 28 October 2003 15:35, eric wrote:

> >Other than that. XML is nice. I suspect it will require a few extra CPU
> > cycles
> >though.
>
> And lots of bandwidth, let's not forget that bandwidth is the bottleneck
> resource.

only if the stream is not compressed and that can be very easy to do using
the zlib which provides API to do this.

Then again, the CPU cylces for compressing is going to kill my P200 hub :)

- -- - -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE/nn70MLjmoUcyZAoRAkiuAKCtzxBeDsrozNHiRrl+84EOiILF0wCfW8/V
PrGV3DE12D1KGhTxc9ktYZM=
=ub6N
-----END PGP SIGNATURE-----

> 6) UTF8 is the encoding of choice for me as well...it should suffice for
> now, although we should keep an eye open for 4-byte unicode support once
> that becomes popular (don't remember if utf8 can encode that or not)

UTF8 supports all kinds of characters (including 2 and 4 byte unicodes) but
use generally less space on western european texts. For asian languages on
the other hand, UTF16 is the better choice considering space/bandwidth.

Is UTF16 compatible with standard strlen [& C°] functions ? When UTF8 needs multiple bytes to encode a character, all the bytes have a value >=128. Does UTF16 has a same kind of feature ? If no, we may see some annoying bytes in the middle of strings (like \0 or even | ).

Eric
DCTC/dchub

I'm not sure which is faster, checking the validity of a message and then

just

pass it through, or recomposing it before sending it along.

Actually, theyre probably both negligible compared to actually doing the
sending of the result. The actual sending/handling of each outgoing
datachunk will be n times more important than how you treat incoming
broadcast messages, where n is the number of users.

---
John Bäckstranmd

Is UTF16 compatible with standard strlen [& C°] functions ?

UTF-8 is US-ASCII (0-127) clean, while UTF-16 is not.

---
John Bäckstrand

4) To maintain some of the binary advantages, I'd suggest going back to the
old FOURCC codes, and have the first letter denote command type. Then BCHA
<from-guid> <text without nick> would be a broadcast chat message, HNIC
<nick> a hub validatenick-type command and CCON <to-guid> <from-guid> be
connecttome. The C type messages would always have to as their first
argument so that the hub could always route them to the correct user even if
it doesn't recognise the command natively (with a setting in the hub which
commands to pass through of course), and parsing is fairly easy as each
command has fixed positions where it can pick up info.

Why not even use only 2 characters (=>26*26 = 676 possibles alpha word). More over, if we are using a GUID instead of nickname, we can even remove some separator because GUID have a constant size. Using this, a private message can be coded (for example)
PMguid1guid2blah blah blah|
which is rather compact and fast to decode.

Eric
DCTC/dchub

UTF-8 is far more compact than UTF-16. UTF8 is a variable length encoding
whose greatest points are to reuse the same encoding for ASCII and to save
space for Western encodings, but it is a bit more complex to handle in
practice. UTF-16 use 2 bytes per characters (and sometimes combines two
pairs), it makes implementation easier, but looks a bit overkill for Western
languages encoding and take twice of memory than UTF-8.

Yves
dchub

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 27 October 2003 10:23, blackclaw@parsoma.net wrote:

CR should not be a separator; I find multiline message support valuable,
for example, for ensuring pasted source code is nicely formatted.

What about escaping it?

Other than that. XML is nice. I suspect it will require a few extra CPU cycles though.

- -- - -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE/nipyMLjmoUcyZAoRAm1KAJ9Qk4ZhVHVkZ/GBd7F5Ao7gHWaP7wCeNBTD
jd8pZ7mHHEM9AoWvpEuRKqI=
=IS9y
-----END PGP SIGNATURE-----

===== Original Message From Direct Connect developers <dcdev@dcportal.net>

=====

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 27 October 2003 10:23, blackclaw@parsoma.net wrote:

CR should not be a separator; I find multiline message support valuable,
for example, for ensuring pasted source code is nicely formatted.

What about escaping it?

You're right. As i said in my first post, i think escaping is the best solution.

Other than that. XML is nice. I suspect it will require a few extra CPU

cycles

though.

And lots of bandwidth, let's not forget that bandwidth is the bottleneck resource.

- --
- -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE/nipyMLjmoUcyZAoRAm1KAJ9Qk4ZhVHVkZ/GBd7F5Ao7gHWaP7wCeNBTD
jd8pZ7mHHEM9AoWvpEuRKqI=
=IS9y
-----END PGP SIGNATURE-----

===== Original Message From Direct Connect developers <dcdev@dcportal.net>

=====

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 27 October 2003 10:23, blackclaw@parsoma.net wrote:

CR should not be a separator; I find multiline message support valuable,
for example, for ensuring pasted source code is nicely formatted.

What about escaping it?

You're right. As i said in my first post, i think escaping is the best solution.

Other than that. XML is nice. I suspect it will require a few extra CPU

cycles

though.

And lots of bandwidth, let's not forget that bandwidth is the bottleneck resource.

- --
- -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE/nipyMLjmoUcyZAoRAm1KAJ9Qk4ZhVHVkZ/GBd7F5Ao7gHWaP7wCeNBTD
jd8pZ7mHHEM9AoWvpEuRKqI=
=IS9y
-----END PGP SIGNATURE-----

Some random thoughts on a new protocol:

1) I like the idea about having 2-3 types of commands, type1 -> hubonly,
type2->a specific client, type3->broadcast. This way, new commands can be
added to clients without upgrading the hubs. Each of these commands would
then have a header for the hub, and the rest of the data for the client
(that the hub can check/parse/deny if needed)

2) I don't like XML. We're running short on bandwidth, so a more
conservative encoding would suit us better imho. CPU btw is afaik not an
issue (dch++ uses about 4% cpu on an average 1500 user hub on a decent
computer, don't remember exact specs, but it's so low that well...), so
perhaps we could use XML + compression if you're fanatic about it, but imho
it's a ferarri when all we need is a fiat...

3) I don't like a pure binary protocol for many reasons, ranging for hard to
debug to hard to interpret on different platforms due to different byte
order. The w3c hasn't put out any binary protocols for a reason, and yet,
these are the most successful ones out there serving millions of clients...

4) To maintain some of the binary advantages, I'd suggest going back to the
old FOURCC codes, and have the first letter denote command type. Then BCHA
<from-guid> <text without nick> would be a broadcast chat message, HNIC
<nick> a hub validatenick-type command and CCON <to-guid> <from-guid> be
connecttome. The C type messages would always have to as their first
argument so that the hub could always route them to the correct user even if
it doesn't recognise the command natively (with a setting in the hub which
commands to pass through of course), and parsing is fairly easy as each
command has fixed positions where it can pick up info.

5) I'd go for the space as separator and CR as end of command, escaping
those as needed with for instance \ (a little like a unix shell...). This to
ensure easy debug-ability...most commands could do without the
space-escaping by keeping guids space-free and placing the spaceneeding
parameter last (i e chat message)

6) UTF8 is the encoding of choice for me as well...it should suffice for
now, although we should keep an eye open for 4-byte unicode support once
that becomes popular (don't remember if utf8 can encode that or not)

In fact, I started writing on a spec for this, taking many good ideas from
jan vidar's dctng.html freely =).

Comments?

P S., This is a lot of work, which means that it'll take time. $Supports for
the hubs, using $Key tricks to avoid sending trash (unknown commands) to the
clients/hubs (for which some hubs kick) is a temporary solution, but one
that works fairly well...NONEXTENDEDPROTOCOL is used by dch++ and ptokax
afaik, and does not interfere with EXTENDEDPROTOCOL, only the beginning of
the key should be parsed (for dc++, parsing was fixed in 0.300). C-C
extended protocol is used by more clients than DC++, specifically to get the
bz lists.
/J

Sorry for intruding. I just wanted to add a personal comment which has completely nothing to do with your current discussion.

The DC protocol is not extensible enough. That was a mistake on my part. I was
young, and didn't give it enough thought. I'd never done anything like that
before - I was 17.

That sure is impressive from you. At 17 I wasn't that creative.

Sorry again.

Richard

On Tuesday, October 28, 2003, at 10:02  AM, eric wrote:

4) To maintain some of the binary advantages, I'd suggest going back to the
old FOURCC codes, and have the first letter denote command type. Then BCHA
<from-guid> <text without nick> would be a broadcast chat message, HNIC
<nick> a hub validatenick-type command and CCON <to-guid> <from-guid> be
connecttome.

Why not even use only 2 characters (=>26*26 = 676 possibles alpha word). More
over, if we are using a GUID instead of nickname, we can even remove some
separator because GUID have a constant size. Using this, a private message
can be coded (for example)
PMguid1guid2blah blah blah|
which is rather compact and fast to decode.

Well, if you use only two characters, you really only have 3*26 command types.  Implicit was the idea that the first character is either H (for client to hub), B (for client to broadcast), or C (for client to client).  This way, if you're not validating commands, the logic required to route the request is pretty simple.  So using only two characters is a little bit of a tradeoff between expansion (and readability) and bandwidth.

Eliminating whitespace between some commands with fixed-length parameters is definitely a worthy idea... though how much we need to reduce the protocol's bandwidth usage is something we should bring up again later.  Certainly we should code and measure a typical DC hub against a similar test protocol hub before we roll it out.

I'm interested in seeing Arne's proposal (as well as those of others).

I see our user count is up to 28 normal + 4 digested users... anyone have something to add? :)

 - Todd
DC++ Contributor

Well, if you use only two characters, you really only have 3*26 command
types.  Implicit was the idea that the first character is either H (for
client to hub), B (for client to broadcast), or C (for client to
client).  This way, if you're not validating commands, the logic
required to route the request is pretty simple. 

I think it is a big waste of space because the 2 main groups of commands are client-hub and client-client, the client to broadcast commands are only a sub-category of client-hub commands and is a very small set of commands (search query & public chat (only ?) ).

Eliminating whitespace between some commands with fixed-length parameters is definitely a worthy idea... though how much we need to reduce the protocol's bandwidth usage is something we should bring up again later.  Certainly we should code and measure a typical DC hub against a similar test protocol hub before we roll it out.

finding the most used commands is not difficult: 1) search query ($Search)
2) connection query ($ConnectToMe)
and if passive users are on the hub:
3) search reply ($SR)
4) reverse connection query ($RevConnectToMe)

Public and private chat commands are probably very insignificant even if they can carry a payload bigger than the 4 previous commands, most of hub have no public chat activity (except kick/ban messages but it is a waste of bandwidth to send such messages to everyone).
Assuming these commands are the most used ones, replacing nickname by GUID will IMHO have a big inpact on the bandwidth because all these commands use one or even two GUID and even a long nickname is shorter the GUID, especially because the GUID is in ASCII and not in binary (like described in a previous mail).

Eric
DCTC/dchub

hi, just a few quick thoughts from my points of view:

- no xml. Too big payload:usefull_data ratio
- no hub-side data compression. There are still low-end hubs running 1000+ users
- for commands format i would prefer either
 a) short fourcc header for every command with command type and data length. Therefore no command separator and no newline/space/or_whatever escaping is needed (I know, the byte-order differences on various OSes but hey, sockets implementation deals with the same problem and it's working fine... htons, ntohl etc...  what's faster? Swapping of two bytes from the header or parsing the data for [end] separator? Not all messages are sent in one piece...)
 or b) let the command is zero-terminated string without length in the header. But then we are back in searching for data-end.
- keeping of own nicks in messages/commands sent to hub? Whatever... In both cases (Checking_and_broadcasting or reconstructing_and_broadcasting) you will have to copy the same amount of data to broadcast it. Just in the first case you will have to perform string comparsion which will fail with spoofed_nicks_commands only, therefore in 99% of cases the strcmp will loop over whole nick. From this point of view the sending of own nicks is rather inconvenient. Two calls to strncpy are faster than one call to strncmp + strncpy. But it all depends on design of certain commands...

lets kick our brains asses ;)

ptaczek.

  a) short fourcc header for every command with command type and data
length. Therefore no command separator and no newline/space/or_whatever
escaping is needed (I know, the byte-order differences on various OSes but
hey, sockets implementation deals with the same problem and it's working
fine... htons, ntohl etc...  what's faster? Swapping of two bytes from the
header or parsing the data for [end] separator? Not all messages are sent
in one piece...)

IMHO, a) is faster than b) and the escape thing is fairly important, especially if we plan to use UTF-16 (no problem with UTF-8).

  a) short fourcc header for every command with command type and data length. Therefore no command separator and no newline/space/or_whatever escaping is needed (I know, the byte-order differences on various OSes but hey, sockets implementation deals with the same problem and it's working fine... htons, ntohl etc...  what's faster? Swapping of two bytes from the header or parsing the data for [end] separator? Not all messages are sent in one piece...)

I have to say I would also like to see a length field for logical packets in
the protocol. Second best would be \0 as a separator.

---
John Bäckstrand

On Friday, November 28, 2003, 16:26:50, Zdenek Stangl wrote:

- no xml. Too big payload:usefull_data ratio

XML is useless for this type of data exchange IMHO.

- no hub-side data compression. There are still low-end hubs running 1000+ users

What about making the compression optional? And, isn't is usually cheaper to
upgrade the hardware than to upgrade the connection type?

Then again, what's the limit on the number of hub users which the *client*
can handle - I remember before upgrading the hardware on my server (used to
be Via C3 533MHz), dcgui would eat all CPU if I connected to 4-6 reasonably
big hubs...

Another thing we should consider is encrypton. Here's why:

- it makes data shaping based on content much harder - the connection won't
 look much different from SSH sessions, or even HTTPS

- it makes the data monitoring impossible - a few weeks ago there were
 rumors that German ISP T-Online would be monitoring P2P transfers of their
 clients, and apparently some people have already gotten into trouble
 because of this

Opinions?

-- 
Jernej Simoncic, ender@ena.si
http://www2.arnes.si/~sopjsimo/
http://deepthought.ena.si/

Experiments should be reproducible. They should all fail in the same way.
        -- Law of Continuity

Another thing we should consider is encrypton. Here's why:

it becomes more and more needed. At this rate, DC protocol will be the only protocol to support the "open a sniffer and see what I do" feature and it is definitely bad.

I have not taken a big look on this but OpenSSL should provide both encryption and compression (except if openssh does compression on its own).

Zdenek Stangl writes:
> hi, just a few quick thoughts from my points of view:
> > - no hub-side data compression. There are still low-end hubs
> running 1000+ users

Like Jernej said, isn't it better to make it optional? That way the
hub owner can decide the policy depending on his CPU or bandwidth. The
hub can simply:
1. Not request compression (For low-CPU hubs)
2. Request compression (For high-end hubs, make the client decide)
3. Require compression (For low-bandwidth hubs)

Like Jernej also said, I think encryption should be considered as
well. There are several good reasons, including the ability to
authenticate on certificate basis (for ops, for example).

> - for commands format i would prefer either
>   a) short fourcc header for every command with command type and
>   data length. Therefore no command separator and no
>   newline/space/or_whatever escaping is needed (I know, the
>   byte-order differences on various OSes but hey, sockets
>   implementation deals with the same problem and it's working
>   fine... htons, ntohl etc...  what's faster? Swapping of two bytes
>   from the header or parsing the data for [end] separator? Not all
>   messages are sent in one piece...)
>   or b) let the command is zero-terminated string without length in
>   the header. But then we are back in searching for data-end.

I do not agree with you. Although binary protocols admittedly have
advantages, it isn't that hard to implement quoting anyway, so why not
simplify debugging and monitoring by using a text-based protocol
instead? When it comes to using CPU power, there are far greater
problems in a DC implementation than protocol parsing. Like searching,
for example.

I do agree that there should be fourcc commands, though, unlike
DN-TNGs idea of using command strings. Fourcc commands simplifies
things for all parties. In most else I agree with DC-TNG, though, for
example having CRLF line terminators for consistency with other
Internet protocols.

Thoughts?

Fredrik Tolf

- for commands format i would prefer either
  a) short fourcc header for every command with command type and
  data length. Therefore no command separator and no
  newline/space/or_whatever escaping is needed (I know, the
  byte-order differences on various OSes but hey, sockets
  implementation deals with the same problem and it's working
  fine... htons, ntohl etc...  what's faster? Swapping of two bytes
  from the header or parsing the data for [end] separator? Not all
  messages are sent in one piece...)
  or b) let the command is zero-terminated string without length in
  the header. But then we are back in searching for data-end.

I do not agree with you. Although binary protocols admittedly have advantages, it isn't that hard to implement quoting anyway, so why notsimplify debugging and monitoring by using a text-based protocol instead? When it comes to using CPU power, there are far greater problems in a DC implementation than protocol parsing. Like searching, for example.

Sure, quoting aint hard. And btw, do you consider the protocol binary
because it has a binary field that specifies length of packets? I dont. I do
still think the content should be text. It wont be hard to debug.

---
John Bäckstrand

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 28 November 2003 18:04, eric wrote:

> Another thing we should consider is encrypton. Here's why:

it becomes more and more needed. At this rate, DC protocol will be the only
protocol to support the "open a sniffer and see what I do" feature and it
is definitely bad.

I have not taken a big look on this but OpenSSL should provide both
encryption and compression (except if openssh does compression on its own).

I'm not sure what OpenSSL can do, but OpenSSH for sure uses zlib for compression.

- -- - -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE/yIyoMLjmoUcyZAoRAo0YAJ43YA2XhttE+1/IHRcD3QhjB1uv8gCfaKTZ
rMCEVhwDDERsywyPZjM3ydo=
=w8lZ
-----END PGP SIGNATURE-----

Zdenek Stangl writes:
> - semi-binary header != binary protocol. All I would like to have is
> the fourcc AND length at the begining of message. Everything else
> remains text-based. With such scheme the debugging is still simple
> + the parsing mechanism would be more generic and fast.

I would have to say that it's _much_ harder to debug if you have to
know in advance how many characters you will type, especially if you
have to type the length in binary format (which for most terminal
emulators is impossible).

Using a pure text based protocols also makes it simpler to use
standard UNIX tools to parse debug information and so on, so at least
in my mind, using a completely text based protocol has many
advantages.

Personally, I would be pleased to see escaping with both backslashes
and double quotes, but that might just be me, or does anyone else
agree?

In any case, when it comes to CPU usage, I believe there are far worse
CPU hogs in a DC client. Stuff like searching and file hashing, and
possibly even translation back and forth from UTF-8 are likely to take
much more CPU than command parsing no matter how much you optimize them
and no matter how many quoting styles you allow in the command
parser. Since it is that way, I'd say that it's preferable to ease
debugging as much as possible before simplifying the command parser
for speed.

I do certainly agree with fourcc commands, though. If not for any
other reason, it allows for faster typing of a command. Maybe it won't
have to be strictly enforced, on the other hand.

That is my opinion. Please tell me if you think I'm wrong about
anything.

Fredrik Tolf

Zdenek Stangl writes:

> - semi-binary header != binary protocol. All I would like to have
> is the fourcc AND length at the begining of message. Everything
> else remains text-based. With such scheme the debugging is still
> simple + the parsing mechanism would be more generic and fast.

I forgot to say, IMHO that would make the parsing algorithm less
generic, since it must treat the command name specially. A generic
algorithm would treat the command just like any other word in the
line.

Fredrik Tolf

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 01 December 2003 13:21, Fredrik Tolf wrote:

I would have to say that it's _much_ harder to debug if you have to
know in advance how many characters you will type, especially if you
have to type the length in binary format (which for most terminal
emulators is impossible).

Using a pure text based protocols also makes it simpler to use
standard UNIX tools to parse debug information and so on, so at least
in my mind, using a completely text based protocol has many
advantages.

Personally, I would be pleased to see escaping with both backslashes
and double quotes, but that might just be me, or does anyone else
agree?

I totally agree ;)
It should be possible to debug using a telnet session, with todays protocol this is for most parts possible if you accept some garbage between the "|"s and disregard the lock/key calculation.

In any case, when it comes to CPU usage, I believe there are far worse
CPU hogs in a DC client. Stuff like searching and file hashing, and
possibly even translation back and forth from UTF-8 are likely to take
much more CPU than command parsing no matter how much you optimize them
and no matter how many quoting styles you allow in the command
parser. Since it is that way, I'd say that it's preferable to ease
debugging as much as possible before simplifying the command parser
for speed.

Well, yes, but the hub shouldn't need to bother doing this in any case, thus
no performance hit on the server side.

- -- - -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE/yzXoMLjmoUcyZAoRAk5HAKC7XbIqr27EddiqI3DH8NuCKW1vbACfUOB9
PN2mgGhlHGPO6xRcoNRdPwk=
=knae
-----END PGP SIGNATURE-----

On Monday 01 December 2003 13:28, Fredrik Tolf wrote:

Zdenek Stangl writes:
 > - semi-binary header != binary protocol. All I would like to have
 > is the fourcc AND length at the begining of message. Everything
 > else remains text-based. With such scheme the debugging is still
 > simple + the parsing mechanism would be more generic and fast.

I forgot to say, IMHO that would make the parsing algorithm less
generic, since it must treat the command name specially. A generic
algorithm would treat the command just like any other word in the
line.

I don't agree with you, even a semi-binary header can speed up stream processing for a hub (and also the client) but only if it is the very thing to come (before any fourcc). Currently, when data arrives, the only way to know if a full command line is received is to search for '|' again and again until the full line is received. If a binary header which contains the bloc size, you just have to check the value with the size of the buffer of incoming data.

The size in the header can also be optimized to be stored in 1 byte (as small as the "|") and for bigger commands, we can use an escape code like 0 or 255 to tell the size is stored in 2 bytes (like in x86 instruction set) (it is probably better to keep 0 as connection pulse to speed up detection of broken connection).

In your previous mail, you also speak about text based protocols being easier to debug because we can use telnet but there is some problem here:
1) you cannot use telnet on DC protocol because telnet sends the command after a "return" and because the end of command is "|" in DC, you will have a LF in the buffer at the beginning of the next command
2) the debug status is only temporary (or it is a very bad protocol with bad programmers).
3) unlike us, clients and hubs always know the size of the data to send before sending them because they have built the command.

About fourcc usage, I am not sure we can gain a lot here because there is some problem to take into account. Comparing 4 characters is always faster than comparing string of unknown size but to be very fast, we must compare the 4 bytes simultaneously (one 32bits value). However, in this case, we must define the endianness of the protocol (big endian is better, like any portable network protocol (TCP, IP,...) else we will have PC clients incompatible with MAC one (ABCD on PC is displayed DCBA on mac). Only a onecc can solve the problem without conversion :)

Eric

> In any case, when it comes to CPU usage, I believe there are far worse
> CPU hogs in a DC client. Stuff like searching and file hashing, and
> possibly even translation back and forth from UTF-8 are likely to take
> much more CPU than command parsing no matter how much you optimize them
> and no matter how many quoting styles you allow in the command
> parser. Since it is that way, I'd say that it's preferable to ease
> debugging as much as possible before simplifying the command parser
> for speed.

Well, yes, but the hub shouldn't need to bother doing this in any case,
thus no performance hit on the server side.

In fact, the choice is even easier. The hub cannot do it. To perform conversion from UTF8 to something else, the hub must know the charset used by each client and there is currently no command allowing a client to send this information to the hub.

About protocol charset, I think (binary or ascii protocol) UTF8 is better then UTF16 because, unlike UTF8, UTF16 conflicts with ASCII 7 bits and this means these characters (the one in conflict) must be escaped which will take more place than UTF8 (DC escape codes are infamous and too big).

Eric

I would like to see particular C/C++ implementation of such statement ;) If you know, that message fourcc tag is 4 bytes long and that the data length bytes are at positions 5 and 6, what's wrong then? First you check the received length against the value in header, if it's ok, then you lookup the command by it's fourcc, process it and then jump right at the begining of next command if there is any... In my opinion it's the same for every single command. Correct me if Im wrong.

pta

-----Original Message-----
From: Fredrik Tolf [mailto:fredrik@dolda2000.com]
Sent: Monday, December 01, 2003 1:28 PM
To: Direct Connect developers
Subject: RE: [dcdev] Re: New Encoding Scheme First

Zdenek Stangl writes:

> - semi-binary header != binary protocol. All I would like to have
> is the fourcc AND length at the begining of message. Everything
> else remains text-based. With such scheme the debugging is still
> simple + the parsing mechanism would be more generic and fast.

I forgot to say, IMHO that would make the parsing algorithm less
generic, since it must treat the command name specially. A generic
algorithm would treat the command just like any other word in the
line.

Fredrik Tolf

eric writes:
> I don't agree with you, even a semi-binary header can speed up
> stream processing for a hub (and also the client) but only if it is
> the very thing to come (before any fourcc). Currently, when data
> arrives, the only way to know if a full command line is received is
> to search for '|' again and again until the full line is
> received. If a binary header which contains the bloc size, you just
> have to check the value with the size of the buffer of incoming
> data.
> > The size in the header can also be optimized to be stored in 1 byte
> (as small as the "|") and for bigger commands, we can use an escape
> code like 0 or 255 to tell the size is stored in 2 bytes (like in
> x86 instruction set) (it is

Or even better, use UTF-8 encoding straight-off. Or simply send
everything over a bzip2 compressed channel.

> probably better to keep 0 as connection pulse to speed up detection
> of broken connection).

Only a semi-binary protocol with line length won't make a very large
difference, since you will still have to search for word
seperators. If you integrate the word splitter with the recieving
logic, you don't have to search through the entire recieve buffer
every time anyway (just the newely recieved data), so a pre-sent
binary length will make only very little difference in that case.

If you want to be able to optimize the command parser to those
lengths, you will need a fully binary protocol, for example having
each command block begin with a word count, and then every word begin
with a character count.

Admittedly, binary protocols do have advantages, it is simply a matter
of which way you want to go. That is a stance we will have to make
before designing a new protocol, and personally, I am in favor of a
text-based protocol.

> In your previous mail, you also speak about text based protocols being easier > to debug because we can use telnet but there is some problem here:
> 1) you cannot use telnet on DC protocol because telnet sends the command after > a "return" and because the end of command is "|" in DC, you will have a LF in > the buffer at the beginning of the next command

That's not true. Windows9x telnet (I haven't run Windows since then,
so I don't know about the new one) sends data as it comes in, and UNIX
telnet also sends data as it arrives on stdin. It is the kernel that
buffers data until it gets CR, and if you send it an EOF character
(^D), the tty driver will flush the line buffer so that telnet sends
it. You can also turn off icanon on the tty before debugging. I have
used that to debug DC sessions, only that I've had to copy the key
from a calculation program.

In any case, I (and I have understood that I'm not the only one) was
hoping that the new protocol would have standard internet CRLF command
terminators, which will solve the problem in any case.

> 2) the debug status is only temporary (or it is a very bad protocol with bad > programmers).

That is a valid argument, though. However, it's not just for
debugging. I find that it's often useful to be able to interact with
software from a remote location for a number of reasons, like to check
status and whatever without having to install a client. It also makes
traffic monitoring easier (unless encryption is used). If used with
compression, it won't use up too much bandwidth either.

> 3) unlike us, clients and hubs always know the size of the data to send before > sending them because they have built the command.

Which is precisely why we can't interact with nodes in that case. But
like I said before, the choice between a text based or binary protocol
is simply a stance we have to take. Shall we hold a poll?

> About fourcc usage, I am not sure we can gain a lot here because there is some > problem to take into account. Comparing 4 characters is always faster than > comparing string of unknown size but to be very fast, we must compare the 4 > bytes simultaneously (one 32bits value). However, in this case, we must > define the endianness of the protocol (big endian is better, like any > portable network protocol (TCP, IP,...) else we will have PC clients > incompatible with MAC one (ABCD on PC is displayed DCBA on mac). Only a onecc > can solve the problem without conversion :)

Like Zdenek said in his first mail, endianess isn't a problem; after
all, that's what the htonX functions are for.

In any case, I would be pleased to see fourcc commands in a text
protocol since they're faster to type. I don't think that we have to
strictly confine ourselves to fourcc commands, on the other hand, but
rather use them by preference and if possible.

On the other hand, with a binary protocol, why use character based
commands at all? Just using numeric IDs would be more than enough. You
can also define an X11-like protocol, adding extensibility to the
protocol by using protocol extension modules.

As a last note, I haven't ever profiled a hub program, so I don't know
how much CPU time is spent in the command parser, but I would imagine
that command multiplexing and retransmission would take far more CPU
time. I don't know how efficient parser algorithms the hubs use
either, but I'd imagine that if you use a stateful reciever, the hub
would have far worse things to worry about. But then again, I haven't
actually profiled a hub, so I don't know for sure. Can anyone who has
profiled a hub give some input here?

There is a third option: to have protocol modules. It seems that the
protocol is to be word-based in any case, so it's just a matter of
writing to word splitters. While I wouldn't like to see this becoming
reality, it would be more generic in the way that the client and hub
could negotiate parser stacks, ie. I would use only textparser when
connecting with telnet, while a client might use
decrypt->decompress->binparser to recieve data. You could also include
multibyte translators that way: decompress->binparser->utf8trans or
textparser->eucjptrans. Just run the stack backwards to send
data. Then again, though, I would not be pleased to see this becoming
reality.

In any case, we need to decide on either a fully binary or fully text
protocol. There's no point at all in having some halfbreed stuff. Like
I've said, I believe in a text based protocol, since I think that both
hubs and clients have far worse things than just command parsing to
care about, but if the majority is against me, I will gracefully bow
and yield. Like I said, shall we hold a poll?

Fredrik Tolf

Zdenek Stangl writes:
> I would like to see particular C/C++ implementation of such
> statement ;) If you know, that message fourcc tag is 4 bytes long
> and that the data length bytes are at positions 5 and 6, what's
> wrong then? First you check the received length against the value
> in header, if it's ok, then you lookup the command by it's fourcc,
> process it and then jump right at the begining of next command if
> there is any... In my opinion it's the same for every single
> command. Correct me if Im wrong.

If you do it that way, yeah. However, since you wanted a word splitter
anyway, I say it's more generic to have the command be part of the
final word array, just like shells and everything else do it. While
your model has packets consisting of three parts (fourcc, length,
data, where the fourcc could just as well be part of the data), my
model only has two elements (length, data). Since the fourcc tags have
the same syntax as the rest of the data anyway, I'd say it's better to
just keep all of it part of the same part of the packet.

That's just my opinion, of course.

Fredrik Tolf

> > -----Original Message-----
> From: Fredrik Tolf [mailto:fredrik@dolda2000.com]
> Sent: Monday, December 01, 2003 1:28 PM
> To: Direct Connect developers
> Subject: RE: [dcdev] Re: New Encoding Scheme First
> > > Zdenek Stangl writes:
> >  > - semi-binary header != binary protocol. All I would like to have
>  > is the fourcc AND length at the begining of message. Everything
>  > else remains text-based. With such scheme the debugging is still
>  > simple + the parsing mechanism would be more generic and fast.
> > I forgot to say, IMHO that would make the parsing algorithm less
> generic, since it must treat the command name specially. A generic
> algorithm would treat the command just like any other word in the
> line.
> > Fredrik Tolf
> > -- >

On Monday, December 1, 2003, 18:02:20, eric wrote:

In your previous mail, you also speak about text based protocols being easier
to debug because we can use telnet but there is some problem here:
1) you cannot use telnet on DC protocol because telnet sends the command after
a "return" and because the end of command is "|" in DC, you will have a LF in
the buffer at the beginning of the next command

Telnet has this problem, but nc (NetCat) doesn't - I successfully used it to
log in to a few hubs which don't do $Key checking, and even managed to chat.
Wasn't pleasant, but it worked.

-- 
Jernej Simoncic, ender@ena.si
http://www2.arnes.si/~sopjsimo/
http://deepthought.ena.si/

Of two possible events, only the undesired one will occur.
      -- Dude's Law of Duality

In any case, I (and I have understood that I'm not the only one) was
hoping that the new protocol would have standard internet CRLF command
terminators, which will solve the problem in any case.

except for multi-line parameters like chat message.

 > 3) unlike us, clients and hubs always know the size of the data to send
 > before sending them because they have built the command.

Which is precisely why we can't interact with nodes in that case. But
like I said before, the choice between a text based or binary protocol
is simply a stance we have to take. Shall we hold a poll?

a poll is probably not a bad idea but I don't know if this will produce something useful :)

Like Zdenek said in his first mail, endianess isn't a problem; after
all, that's what the htonX functions are for.

and that's also why big endian must be used because the htonX functions does nothing on big endian computer.

In any case, we need to decide on either a fully binary or fully text
protocol. There's no point at all in having some halfbreed stuff. Like
I've said, I believe in a text based protocol, since I think that both
hubs and clients have far worse things than just command parsing to
care about, but if the majority is against me, I will gracefully bow
and yield. Like I said, shall we hold a poll?

For the hub efficiency and command size (especially if we switch from nickname to ID), a binary protocol will have better performance in processing speed (no text<=>binary conversion) but also provide possibilities text protocol cannot reach like command with optional (or newly created) parameters or even command with constant parameter with different parameter type (imagine a $Search allowing either a hash ID or a pattern). However, binary protocols also have their drawbacks.

Eric

...

algorithm would treat the command just like any other word in the
line.

I don't agree with you, even a semi-binary header can speed up stream processing for a hub (and also the client) but only if it is the very thing to come (before any fourcc). Currently, when data arrives, the only way to know if a full command line is received is to search for '|' again and again until the full line is received. If a binary header which contains the bloc size, you just have to check the value with the size of the buffer of incoming data.

That's my point ;)

About fourcc usage, I am not sure we can gain a lot here because there is some problem to take into account. Comparing 4 characters is always faster than comparing string of unknown size but to be very fast, we must compare the 4 bytes simultaneously (one 32bits value). However, in this case, we must define the endianness of the protocol (big endian is better, like any portable network protocol (TCP, IP,...) else we will have PC clients incompatible with MAC one (ABCD on PC is displayed DCBA on mac). Only a onecc can solve the problem without conversion :)

Because of that a protocol is called protocol ;) Once you design the format (in this case the order of bytes in fourcc tag and order of length bytes) then it becomes a rule which needs certain implementations on certain platforms/OSes ;) Lets focus on protocol design, not the API.

Eric

Zdenek

eric writes:
> > > In any case, I (and I have understood that I'm not the only one) was
> > hoping that the new protocol would have standard internet CRLF command
> > terminators, which will solve the problem in any case.
> > except for multi-line parameters like chat message.

However, that's what quoting is good foor.

> > >  > 3) unlike us, clients and hubs always know the size of the data to send
> >  > before sending them because they have built the command.
> >
> > Which is precisely why we can't interact with nodes in that case. But
> > like I said before, the choice between a text based or binary protocol
> > is simply a stance we have to take. Shall we hold a poll?
> > a poll is probably not a bad idea but I don't know if this will produce > something useful :)

Actually, I was thinking the same. ;-)

Maybe we should create some test data, a binary parser and a text
parser, and compare their performance?

In any case, I think we should decide on a protocol ASAP, so that we
can begin to decide on the internals of it. How about we do that test,
present the data somewhere on the net and let people choose?

> > In any case, we need to decide on either a fully binary or fully text
> > protocol. There's no point at all in having some halfbreed stuff. Like
> > I've said, I believe in a text based protocol, since I think that both
> > hubs and clients have far worse things than just command parsing to
> > care about, but if the majority is against me, I will gracefully bow
> > and yield. Like I said, shall we hold a poll?
> > For the hub efficiency and command size (especially if we switch
> from nickname to ID), a binary protocol will have better
> performance in processing speed (no text<=>binary conversion) but
> also provide possibilities text protocol cannot reach like command
> with optional (or newly created) parameters or even command with
> constant parameter with different parameter type (imagine a $Search
> allowing either a hash ID or a pattern). However, binary protocols
> also have their drawbacks.

Why do you mean that text protocols cannot handle optional or variant
parameters? I see no reason for that.

As I see it, the only thing that speaks in favor of a binary protocol
is parsing speed. That is also why I suggest the test I mentioned. I
would also like to see the result of profiling a complete hub
implementation, to see how much CPU time command parsing takes in
proportion to the rest of the stuff. If command parsing takes like 10%
or less anyway, I think a binary protocol won't do enough difference
to be considered advantageous.

Fredrik Tolf

Admittedly, binary protocols do
have advantages, it is simply a
matter of which way you want to
go. That is a stance we will
have to make before designing a
new protocol, and personally, I
am in favor of a text-based
protocol.

I agree with this. Also, note that parsing imo is a tiny, tiny fraction of
cpu-time spent by the hub. What takes time in a hub is, unsurprisingly, the
broadcasting.
Quoting, semi-binary, fully binary or fully text are details that wont
really affect the performance or semantics of the protocol. Sadly, everyone
has preferences when it comes to these details, and reaching an agreement is
hard if not impossible. I say, let Arne decide the format of the protocol
(Why? I think he has shown us that he is good at ignoring what others think
about things/deciding things for others), but let us all discuss what the
protocol should actually _do_.

---
John Bäckstrand

Oh now I got it ;) The reason why I've wanted the fourcc first was that the fourcc tag would also act as command-start flag and as such it should start always with the same pattern because the data length bytes will always vary. With such header there is also no need for data-end separator and the 0 byte could be used for other tasks such as heart-beat signaling. To be a little more exact, here is my rough idea of the message packet format:

|- 6_bytes_hdr -| |---data--->
FF FF 00 XX LL LH DA TA .. ..

FF FF - the data start flag
00 XX - command code
LL - low byte of length word
LH - high byte of length word
(data length includes the header length - simply length of data used in send())

The 00 byte of the command code seems to be redundant, but reading from memory without explicit type conversion is always better. With this model Im able to read bytes 2-5 at once into single int variable, and have data-length by simple masking the value by 0x0000FFFF or get the command code with 0xFFFF0000 mask. Then by #defining commands from 0x00010000 up, there is also no need for bit-shifting, if I could dig into possible implementation details here :o)
The data header is always 6 bytes long, so there is no big deal with searching for command data begin.

Just for sure: Im thinking about this all from a hub-coder perspective. For me it's always big advantage to be able look-up a command quickly and know the data length without the need to search for command separator (in other words: loop_over_entire_data_block_and_look_for_the_pipe_by_comparing_each_byte_with_0x7C. There is no kernel_based solution under windows systems). I don't care of UTF encoding or other client-side issues. All Im doing in most cases is receiving-data -> processing-data (with minimal alternation) -> transmitting/broadcasting-data and I have to do it really fast to serve more than thousand of simultaneous connections. Therefore the message-packet length and unified command identificators would be of great help (not just) to me.

Comment it please, Im willing to discuss this matter deeply and I would like to find good compromise which would satisfy both the hub and the client coders.

Zdenek.

-----Original Message-----
From: Fredrik Tolf [mailto:fredrik@dolda2000.com]
Sent: Monday, December 01, 2003 7:44 PM
To: Direct Connect developers
Subject: RE: [dcdev] Re: New Encoding Scheme First

Zdenek Stangl writes:
> I would like to see particular C/C++ implementation of such
> statement ;) If you know, that message fourcc tag is 4 bytes long
> and that the data length bytes are at positions 5 and 6, what's
> wrong then? First you check the received length against the value
> in header, if it's ok, then you lookup the command by it's fourcc,
> process it and then jump right at the begining of next command if
> there is any... In my opinion it's the same for every single
> command. Correct me if Im wrong.

If you do it that way, yeah. However, since you wanted a word splitter
anyway, I say it's more generic to have the command be part of the
final word array, just like shells and everything else do it. While
your model has packets consisting of three parts (fourcc, length,
data, where the fourcc could just as well be part of the data), my
model only has two elements (length, data). Since the fourcc tags have
the same syntax as the rest of the data anyway, I'd say it's better to
just keep all of it part of the same part of the packet.

That's just my opinion, of course.

Fredrik Tolf

> > -----Original Message-----
> From: Fredrik Tolf [mailto:fredrik@dolda2000.com]
> Sent: Monday, December 01, 2003 1:28 PM
> To: Direct Connect developers
> Subject: RE: [dcdev] Re: New Encoding Scheme First
> > > Zdenek Stangl writes:
> >  > - semi-binary header != binary protocol. All I would like to have
>  > is the fourcc AND length at the begining of message. Everything
>  > else remains text-based. With such scheme the debugging is still
>  > simple + the parsing mechanism would be more generic and fast.
> > I forgot to say, IMHO that would make the parsing algorithm less
> generic, since it must treat the command name specially. A generic
> algorithm would treat the command just like any other word in the
> line.
> > Fredrik Tolf
> > -- >

John Bäckstrand writes:
> >Admittedly, binary protocols do have advantages, it is simply a
> >matter of which way you want to go. That is a stance we will have
> >to make before designing a new protocol, and personally, I am in
> >favor of a text-based protocol.
> > I agree with this. Also, note that parsing imo is a tiny, tiny fraction of
> cpu-time spent by the hub. What takes time in a hub is, unsurprisingly, the
> broadcasting.
That is precisely what I thought. Since binary protocols have almost
no advantages except speed (and possibly security, since sloppy
programmers have been shown to put in some bugs along with quoting
code) over text protocols, I would very much like to see a text
protocol being implemented.

However, it would be nice to have someone profile a hub to verify for
sure that that's true. Has anyone already done so, or is someone up to
the task? I would prefer not to do it, since I have no hub software
installed.

Fredrik Tolf

Zdenek Stangl writes:
> Oh now I got it ;) The reason why I've wanted the fourcc first was
> that the fourcc tag would also act as command-start flag and as
> such it should start always with the same pattern because the data
> length bytes will always vary. With such header there is also no
> need for data-end separator and the 0 byte could be used for other
> tasks such as heart-beat signaling. To be a little more exact, here
> is my rough idea of the message packet format:
> > |- 6_bytes_hdr -| |---data--->
> FF FF 00 XX LL LH DA TA .. ..
> > FF FF - the data start flag
> 00 XX - command code
> LL - low byte of length word
> LH - high byte of length word
> (data length includes the header length - simply length of data used
> in send())

Might I ask what you want the FF FF start word for?

> The 00 byte of the command code seems to be redundant, but reading
> from memory without explicit type conversion is always better. With
> this model Im able to read bytes 2-5 at once into single int
> variable, and have data-length by simple masking the value by
> 0x0000FFFF or get the command code with 0xFFFF0000 mask. Then by
> #defining commands from 0x00010000 up, there is also no need for
> bit-shifting, if I could dig into possible implementation details
> here :o) The data header is always 6 bytes long, so there is no big
> deal with searching for command data begin.
>
> Just for sure: Im thinking about this all from a hub-coder
> perspective. For me it's always big advantage to be able look-up a
> command quickly and know the data length without the need to search
> for command separator (in other words:
> loop_over_entire_data_block_and_look_for_the_pipe_by_comparing_each_byte_with_0x7C. There
> is no kernel_based solution under windows systems). I don't care of
> UTF encoding or other client-side issues. All Im doing in most
> cases is receiving-data -> processing-data (with minimal
> alternation) -> transmitting/broadcasting-data and I have to do it
> really fast to serve more than thousand of simultaneous
> connections. Therefore the message-packet length and unified
> command identificators would be of great help (not just) to me.

I just find it a bit unnecessary that you are into it as far as
eliminating a single instruction for bit-shifting, when you still need
to broadcast packets to 1000+ users, going through the kernel's TCP
assembly code and everything. To me, that optimization seems far less
than necessary, since your program will be blocking most of the time
anyway, waiting for the kernel to flush network data.

Especially so since you still have to word-split the data to process
it.

Fredrik Tolf

- compression should be optional, no doubts about it ;)
- semi-binary header != binary protocol. All I would like to have is the fourcc AND length at the begining of message. Everything else remains text-based. With such scheme the debugging is still simple + the parsing mechanism would be more generic and fast.

-----Original Message-----
From: Fredrik Tolf [mailto:fredrik@dolda2000.com]
Sent: Friday, November 28, 2003 8:18 PM
To: Direct Connect developers
Subject: Re: [dcdev] Re: New Encoding Scheme First

Zdenek Stangl writes:
> hi, just a few quick thoughts from my points of view:
> > - no hub-side data compression. There are still low-end hubs
> running 1000+ users

Like Jernej said, isn't it better to make it optional? That way the
hub owner can decide the policy depending on his CPU or bandwidth. The
hub can simply:
1. Not request compression (For low-CPU hubs)
2. Request compression (For high-end hubs, make the client decide)
3. Require compression (For low-bandwidth hubs)

Like Jernej also said, I think encryption should be considered as
well. There are several good reasons, including the ability to
authenticate on certificate basis (for ops, for example).

> - for commands format i would prefer either
>   a) short fourcc header for every command with command type and
>   data length. Therefore no command separator and no
>   newline/space/or_whatever escaping is needed (I know, the
>   byte-order differences on various OSes but hey, sockets
>   implementation deals with the same problem and it's working
>   fine... htons, ntohl etc...  what's faster? Swapping of two bytes
>   from the header or parsing the data for [end] separator? Not all
>   messages are sent in one piece...)
>   or b) let the command is zero-terminated string without length in
>   the header. But then we are back in searching for data-end.

I do not agree with you. Although binary protocols admittedly have
advantages, it isn't that hard to implement quoting anyway, so why not
simplify debugging and monitoring by using a text-based protocol
instead? When it comes to using CPU power, there are far greater
problems in a DC implementation than protocol parsing. Like searching,
for example.

I do agree that there should be fourcc commands, though, unlike
DN-TNGs idea of using command strings. Fourcc commands simplifies
things for all parties. In most else I agree with DC-TNG, though, for
example having CRLF line terminators for consistency with other
Internet protocols.

Thoughts?

Fredrik Tolf

Zdenek Stangl writes:
> > Might I ask what you want the FF FF start word for?
> > for the case of total data distortion. I think it's always better
> to have some 'anchor' for cases, when data goes inconsistent or
> mangled, than none.
> > > I just find it a bit unnecessary that you are into it as far as
> > eliminating a single instruction for bit-shifting, when you still need
> > to broadcast packets to 1000+ users, going through the kernel's TCP
> > assembly code and everything. To me, that optimization seems far less
> > than necessary, since your program will be blocking most of the time
> > anyway, waiting for the kernel to flush network data.
> > > > Especially so since you still have to word-split the data to process
> > it.
> > Doing word-splitting for every incomming data on the hub-side isn't
> really good idea. For current DC protocol, Im having custom parsing
> routine for every single command, looking only for tokens or
> specific parts that I really need for the processing. Im also
> avoiding data copying/moving as much as possible and believe me,
> all this has helped to gain performance of ptokax by approximately
> 15% in 0.330 version in comparsion with currenlty available 0.326,
> which uses string classes and copies data a lot.

Well, I can understand that there was a huge performance gain if you
were using C++ string classes before. Those don't only move and copy
data, they even malloc during operation, so it's no wonder you
experienced a boost when abandoning them. However, it's fairly easy to
manipulate text data in place in C to do word splitting and
dequoting. The only thing that demands data moving is backslash
removal, but see below.

> And that's just the parsing mechanism. All sockets are in
> non-blocking mode - when underlaying tcp kernel wouldblock, im
> buffering data by myself - no blocking, no threads.

Oh yes, sorry for being unclear. I hadn't expected you to use blocking
sockets. I was mainly thinking about the time you have to wait until
sending again.

> There is still something missing in our discussion, to have it
> constructive. Try to come out with alternatives, if you can't agree
> with my draft, please.

OK, for some reason I thought my protocol idea was clear. Now that you
mention it, I don't know where I got that from... ;-)

I suggest using a line-based protocol, with CRLF line termination (the
CRLF can also be thought of as doing the same job as the FF FF that
you suggest, except that it can also be quoted). Within lines, words
are seperated by whitespace (I was planning on simply using isspace(3)
for detecting whitespace, but I can agree to only allowing ASCII 32
spaces instead for efficiency). As for quoting, I suggest allowing
both double quotes and backslash escaping, for good reasons. Double
quotes escaping is limited to only include whole words - ie. no
sub-word quoting like a"b c"d instead of "ab cd". Backslashes can
quote anything.

The only thing that would require data movement is backslash
removal. As for all else, just insert NULs where you want them. Since
double quotes can quote everything but themselves, and words are
likely to not very often contain double quotes or backslashes,
backslash removal will probably be a rather rare procedure (yes, I
want pathnames to consist of slashes, not backslashes). Also, if you
want to optimize it on the hub side, you can simply choose not to
dequote words that you don't need to look at.

Also, if you must insist on a binary protocol, I for one would like to
see that it handles all the words in a binary manner as well. Each
command would begin with (possibly a start word,) a word count, and
then having each word begin with a character count. Example (big
endian, 16-bit counters, no start word):

00 03 00 04 "Chat" 00 14 (20 character UUID) 00 0E "Hi everyone!\r\n"

Of course, in both protocols, all human readable words should be
encoded in UTF-8, if that wasn't already obvious.

Fredrik Tolf

To be a little more exact, here is my rough idea of the message packet

format:

|- 6_bytes_hdr -| |---data--->

FF FF 00 XX LL LH DA TA .. ..

FF FF - the data start flag
00 XX - command code
LL - low byte of length word
LH - high byte of length word
(data length includes the header length - simply length of data used in
send())

...

Just for sure: Im thinking about this all from a hub-coder perspective. For
me it's always big advantage to be able look-up a command quickly and know
the data length without the need to search for command separator (in other
words:

I totally agree with you on the header format and I also think the data (which are like command parameter) should have a similar structure.

loop_over_entire_data_block_and_look_for_the_pipe_by_comparing_each_byte_wi
th_0x7C. There is no kernel_based solution under windows systems). I don't

I don't think unix has such thing and in any case, kernel_based only means the kernel does it. It is easier to let the kernel or a library to it but it is always slower to do something comparing to not to do it.

care of UTF encoding or other client-side issues. All Im doing in most
cases is receiving-data -> processing-data (with minimal alternation) ->
transmitting/broadcasting-data and I have to do it really fast to serve
more than thousand of simultaneous connections. Therefore the
message-packet length and unified command identificators would be of great
help (not just) to me.

I'm 100% with you :)

Comment it please, Im willing to discuss this matter deeply and I would
like to find good compromise which would satisfy both the hub and the
client coders.

My opinion only engages me as both hub and client coder :)

Eric

On Tuesday 02 December 2003 00:45, Zdenek Stangl wrote:

> Might I ask what you want the FF FF start word for?

for the case of total data distortion. I think it's always better to have
some 'anchor' for cases, when data goes inconsistent or mangled, than none.

most of binary protocols handling data corruption (PES stream, mp3, ...) has this kind of pattern.

Doing word-splitting for every incomming data on the hub-side isn't really
good idea. For current DC protocol, Im having custom parsing routine for
every single command, looking only for tokens or specific parts that I
really need for the processing. Im also avoiding data copying/moving as
much as possible and believe me, all this has helped to gain performance of
ptokax by approximately 15% in 0.330 version in comparsion with currenlty
available 0.326, which uses string classes and copies data a lot. And
that's just the parsing mechanism. All sockets are in non-blocking mode -
when underlaying tcp kernel wouldblock, im buffering data by myself - no
blocking, no threads.

That's right, I also use this kind of tips on my hub. Without this, you have very bad performance.

There is still something missing in our discussion, to have it
constructive. Try to come out with alternatives, if you can't agree with my
draft, please.

Yes, it is a good idea. Instead of making on poll for a text-based protocol or a binary-based one, why not making a draft of both and proposing it here ?

Eric

On Tuesday 02 December 2003 11:16, Jacek Sieka wrote:

> About fourcc usage, I am not sure we can gain a lot here
> because there is some
> problem to take into account. Comparing 4 characters is
> always faster than
> comparing string of unknown size but to be very fast, we must
> compare the 4
> bytes simultaneously (one 32bits value). However, in this
> case, we must
> define the endianness of the protocol (big endian is better, like any
> portable network protocol (TCP, IP,...) else we will have PC clients
> incompatible with MAC one (ABCD on PC is displayed DCBA on
> mac). Only a onecc
> can solve the problem without conversion :)

Uhm, fourcc, four character code, stress on _character_ (one byte), i e no
endianness...

Yes I know, 1 byte is smaller in size but provides less possible command (how many commands DC protocol has ?).

Eric

 > And that's just the parsing mechanism. All sockets are in
 > non-blocking mode - when underlaying tcp kernel wouldblock, im
 > buffering data by myself - no blocking, no threads.

Oh yes, sorry for being unclear. I hadn't expected you to use blocking
sockets. I was mainly thinking about the time you have to wait until
sending again.

waiting ??? waiting => blocking socket. non-blocking socket => polling and polling costs nothing

Also, if you must insist on a binary protocol, I for one would like to
see that it handles all the words in a binary manner as well. Each
command would begin with (possibly a start word,) a word count, and
then having each word begin with a character count. Example (big
endian, 16-bit counters, no start word):

00 03 00 04 "Chat" 00 14 (20 character UUID) 00 0E "Hi everyone!\r\n"

Of course, in both protocols, all human readable words should be
encoded in UTF-8, if that wasn't already obvious.

I think this is the only thing on what everyone agrees :)

A note on scanning for eol tokens, whatever they may be, come on, it doesn't
take any cpu...incoming data is rather small, even on a big hub, dch++ for
instance doesn't take more than 4-5% cpu overall on a 1500 user hub (1ghz
cpu...), and I can assure you that less than 0.01% of that goes to scanning
for the end of line...therefore, imho, no command lengths in
binary...although I dislike the fact that \r\n are two characters, I tend to
agree that it's good following standards, and since millions of web servers
etc can handle millions of connections, I don't really see the problem...I
mean, wake up, turn on your profiler and have a look at where the cpu usage
goes, I can promise you it isn't strcmp in onincomingdata...

/J

About fourcc usage, I am not sure we can gain a lot here because there is some problem to take into account. Comparing 4 characters is always faster than comparing string of unknown size but to be very fast, we must compare the 4 bytes simultaneously (one 32bits value). However, in this case, we must define the endianness of the protocol (big endian is better, like any portable network protocol (TCP, IP,...) else we will have PC clients incompatible with MAC one (ABCD on PC is displayed DCBA on mac). Only a onecc can solve the problem without conversion :)

Uhm, fourcc, four character code, stress on _character_ (one byte), i e no
endianness...

/J

----- Original Message -----
From: "Jacek Sieka" <j_s@telia.com>
To: "'Direct Connect developers'" <dcdev@dcportal.net>
Sent: Tuesday, December 02, 2003 10:56 AM
Subject: RE: [dcdev] Re: New Encoding Scheme First

A note on scanning for eol tokens, whatever they may be, come on, it

doesn't

take any cpu...incoming data is rather small, even on a big hub, dch++ for
instance doesn't take more than 4-5% cpu overall on a 1500 user hub (1ghz
cpu...), and I can assure you that less than 0.01% of that goes to

scanning

for the end of line...therefore, imho, no command lengths in
binary...although I dislike the fact that \r\n are two characters, I tend

to

agree that it's good following standards, and since millions of web

servers

etc can handle millions of connections, I don't really see the problem...I
mean, wake up, turn on your profiler and have a look at where the cpu

usage

goes, I can promise you it isn't strcmp in onincomingdata...

That's what Im doing almost whole day.. And Im close to leave the
semi-binary garbage once for ever .)
Z.

/J

Might I ask what you want the FF FF start word for?

for the case of total data distortion. I think it's always better to have some 'anchor' for cases, when data goes inconsistent or mangled, than none.

I just find it a bit unnecessary that you are into it as far as
eliminating a single instruction for bit-shifting, when you still need
to broadcast packets to 1000+ users, going through the kernel's TCP
assembly code and everything. To me, that optimization seems far less
than necessary, since your program will be blocking most of the time
anyway, waiting for the kernel to flush network data.

Especially so since you still have to word-split the data to process
it.

Doing word-splitting for every incomming data on the hub-side isn't really good idea. For current DC protocol, Im having custom parsing routine for every single command, looking only for tokens or specific parts that I really need for the processing. Im also avoiding data copying/moving as much as possible and believe me, all this has helped to gain performance of ptokax by approximately 15% in 0.330 version in comparsion with currenlty available 0.326, which uses string classes and copies data a lot. And that's just the parsing mechanism. All sockets are in non-blocking mode - when underlaying tcp kernel wouldblock, im buffering data by myself - no blocking, no threads.

There is still something missing in our discussion, to have it constructive. Try to come out with alternatives, if you can't agree with my draft, please.

Fredrik Tolf

Zdenek.

John Bäckstrand wrote:

I say, let Arne decide the format of the protocol
(Why? I think he has shown us that he is good at ignoring what others think
about things/deciding things for others), but let us all discuss what the
protocol should actually _do_.

This was an excellent point when I first read it, it's time I responded.

These are some of the ideas that I'd like to be possible under any new protocol:

   * Metadata returning in/with search search results
   * Metadata about files contained in share lists
   * Searching by metadata (including at the least the ability to
     return a mp3 hit when only some portion of the ID3 tag is matched)
     (DC++ will filter these results currently)
   * Partial file sharing (with arbitrary ranges of the file completed
     - not necessarily just the first X bytes)
   * Upload queues (yes more than one, and on arbitrary file/user
     characteristics [like a partial file queue that would push through
     partial file requests quickly])
   * Per-client karma/"ratings" system for rewarding downloads
   * Source exchange, at least of clients you've seen on the same hub
     (to avoid information leakages about clients you've seen on
     non-public hubs)
   * Hashing compatible with one of the web-based metadata
     concentrators - like bitzi.com

These are bullet items... most of them have been fleshed out.  Don't bother trying to argue against them; these are my suggestions on what the protocol should do.

From the ADC drafts so far, none of these features are ruled out by it.

- Todd

Todd Pederzani wrote:

Don't bother trying to argue against them; these are my suggestions on what the protocol should do.

This is phrased ever-so-badly.  /Please /add your own suggestions, just don't get bogged down in arguing against mine.  I know some people participating in DC forums love DC just the way it is, and karma and upload queues (and queue rotation) would somehow ruin it for them.  All of my suggestions come from other P2P clients (including BitTorrent, but eMule and Shareaza in particular), and I think some of their lessons can be applied (or should be possible) in any new protocol.

Like Sandos said, the emphasis should be on what we want the protocol to *do* rather than it should *look*, shouldn't it?

- Todd