Hm, just to see if anyone's still alive, I'm posting an ADC spec that is almost finished, and would appreciate any comments you might have...this is most probably the last chance to improve on it, so make sure to take it if you care for the protocol that DC++ at least will move towards... Much of it is already implemented in dc++ and I've heard of a few hub dev projects picking up on it so it's time to get the show off the road I think... Oh, and there's the private dc developers hub that's still up and running, drop me a mail if you want an account... /J
--
Hm, just to see if anyone's still alive, I'm posting an ADC spec that is almost finished, and would appreciate any comments you might have...this is most probably the last chance to improve on it, so make sure to take it if you care for the protocol that DC++ at least will move towards...
Much of it is already implemented in dc++ and I've heard of a few hub dev projects picking up on it so it's time to get the show off the road I think...
Oh, and there's the private dc developers hub that's still up and running, drop me a mail if you want an account...
> I recently went through the whole 0.8 draft (but I only skimmed the > attachment you sent). It looks good, but one feature I am missing is; > if one client via a hub wants to request a connection (CTM) and the > requesting > party cannot connect (connection refused, firewalled/timeout > or whatever).
A DSTA will do just that, adding an error type for it probably makes sense...
> There should be a notify-message that the connection could not be > established. > (Only if the connection attempt had a token). > This can for example be done with a RCM using the same token > back to the > origin. > This way both parties can be aware that the connection cannot > be made, not > just > the connecting party. And thus, this aids network diagnostics.
> Extending this, a passive client can send a RCM to another > passive client > which will answer with RCM and the same token. Now, both > clients know that > they are passive and cannot communicate directly.
Client passivity can already be deduced from the INF Another thing I'm considering (suggested by some dcdev people =) is a publish-subscriber message type that gets a channel id and where the messages are only sent out to the clients that have subscribed. This could be used by extensions for instance to only send out a particular command to clients supporting a particular feature...the only thing that worries me is that it potentially makes the hub a bit more complex...on the other hand, almost all message passing protocols have these three types, direct, subscriber and broadcast so... /J --
I recently went through the whole 0.8 draft (but I only skimmed theA DSTA will do just that, adding an error type for it probably makes sense...
attachment you sent). It looks good, but one feature I am missing is;
if one client via a hub wants to request a connection (CTM) and the
requesting
party cannot connect (connection refused, firewalled/timeout
or whatever).
(...)
Client passivity can already be deduced from the INF
Another thing I'm considering (suggested by some dcdev people =) is a publish-subscriber message type that gets a channel id and where the messages are only sent out to the clients that have subscribed. This could be used by extensions for instance to only send out a particular command to clients supporting a particular feature...the only thing that worries me is that it potentially makes the hub a bit more complex...on the other hand, almost all message passing protocols have these three types, direct, subscriber and broadcast so...
Hello all, If adc finally is moving into 'stable' state, I have a few concerns... First of all 2.1 states: "All text data in the protocol that is not entered by the user, including protocol names, extensions etc may only contain viewable characters that may be encoded by one byte in the UTF-8 encoding (ASCII codes 33-127). Although the protocol is case-sensitive, names distinguished only by case are disallowed (upper case is preferred)." Here, "names" are nick-names I suppose. Should they (as they once Are "entered by the user") be able to contain multi-byte UTF-8 chars or single-byte > 127? I believe that's what "All texts must be sent as UTF-8, including file lists, searches, nick’s, hub lists etc." means. In this case, hubs must include full UTF-8 supported case insensitive matching I suppose... This is not a complaint or anything; I just wanted to be clear of the issue that true Unicode enabled string functions are required in ADC generally. As in contrast to 'regular DC' were currently, any non-utf-8 enabled string function will (probably) do (no matter what internal string encoding is used, for instance with dc++ gui being Unicode nowadays), since UTF-8 is a nice encoding, and since DC doesn't support anything but 'ACP'. So relying on current implementation when it comes to string functions isn't a good idea I suppose. (Just a hint for any "ADC-developer" if I have got it all right.) Secondly, I've said this before (far over a year ago) I like the idea of encrypting the traffic, and I barely see theoretical limitations to this, even though p2p data speeds are 'high' nowadays. I know of certain internet providers (universities for instance), that have taken action against users simply because of filenames in shares (that are being sent clear-text as bz2 filelists). They are using programs analysing this kind of traffic automatically (you all probably know this, I know). This is not 'our' problem, but I think it's not more than fair to try to develop a 'native' support for an encryption base in the main protocol. P2P encryption is probably not very difficult with individual key-pairs or anything similar, but what's probably not so easily done is search replies (and eventual client-hub encryption). Maybe something like the GPA/PAS-routine could be done for client-hub ciphering to avoid usual encryption keys (if this is heavy for hubs to generate/encrypt/decrypt). If encryption becomes native, a key-propagation in the search message should be easily implemented, so the clients encrypt the search reply with the given key. Note; if this is Not natively implemented in the base protocol, then future implementations are a lot more difficult, since there is no exchange of "extended client features" in the usual client2hub communication. /There are no way to know another clients public key/ in other words. This could be sent in a wider INF for sure, but I'm not certain it would be any more efficient, so "making room" for this natively could be a good idea. I have the feeling that encryption isn't something most of you are interested in, but I still want to know how you feel about it. And I would like feedback on my urge for the native support of (I agree, maybe _future_) key exchanges in the base messages presented by the protocol. /Gustaf aka 'Opera' (and yes, I'll convert from outlook to thunderbird one day) -----Ursprungligt meddelande----- Från: dcdev-bounces@dcportal.net [mailto:dcdev-bounces@dcportal.net] För Jacek Sieka Skickat: den 28 december 2004 17:11 Till: 'Direct Connect developers' Ämne: RE: [dcdev] Anyone still alive?
> I recently went through the whole 0.8 draft (but I only skimmed the > attachment you sent). It looks good, but one feature I am missing is; > if one client via a hub wants to request a connection (CTM) and the > requesting > party cannot connect (connection refused, firewalled/timeout > or whatever).
A DSTA will do just that, adding an error type for it probably makes sense...
> There should be a notify-message that the connection could not be > established. > (Only if the connection attempt had a token). > This can for example be done with a RCM using the same token > back to the > origin. > This way both parties can be aware that the connection cannot > be made, not > just > the connecting party. And thus, this aids network diagnostics.
> Extending this, a passive client can send a RCM to another > passive client > which will answer with RCM and the same token. Now, both > clients know that > they are passive and cannot communicate directly.
Client passivity can already be deduced from the INF Another thing I'm considering (suggested by some dcdev people =) is a publish-subscriber message type that gets a channel id and where the messages are only sent out to the clients that have subscribed. This could be used by extensions for instance to only send out a particular command to clients supporting a particular feature...the only thing that worries me is that it potentially makes the hub a bit more complex...on the other hand, almost all message passing protocols have these three types, direct, subscriber and broadcast so... /J --
Here, "names" are nick-names I suppose. Should they (as they once Are
"entered by the user") be able to contain multi-byte UTF-8 chars or
single-byte > 127? I believe that's what "All texts must be sent as UTF-8,
including file lists, searches, nick’s, hub lists etc." means.
In this case, hubs must include full UTF-8 supported case insensitive
matching I suppose... <... snip elaboration ...>
Secondly, I've said this before (far over a year ago) I like the idea
of encrypting the traffic, and I barely see theoretical limitations
to this, even though p2p data speeds are 'high' nowadays. <... snip
...> Maybe something like the
GPA/PAS-routine could be done for client-hub ciphering to avoid usual
encryption keys (if this is heavy for hubs to generate/encrypt/decrypt). If
encryption becomes native, a key-propagation in the search message should be
easily implemented, so the clients encrypt the search reply with the
given key. Note; if this is Not natively implemented in the base
protocol, then future
implementations are a lot more difficult, since there is no exchange
of "extended client features" in the usual client2hub communication.
There are no way to know another clients public key/ in other words. This could
be sent in a wider INF for sure, but I'm not certain it would be any
more efficient, so "making room" for this natively could be a good
idea.
I have the feeling that encryption isn't something most of you are interested in, but I still want to know how you feel about it. And I
would like feedback on my urge for the native support of (I agree,
maybe _future_) key exchanges in the base messages presented by the protocol.
XML markup is case-sensitive because the cost of monocasing in
Unicode is horrible, horrible, horrible. Go look at the source code in your
local java or .Net library.
Also, not only is it expensive, it's just weird. The upper-case of é
is different in France and Quebec, and the lower-case of 'I' is
different here and in Turkey.
XML was monocase until quite late in its design, when we ran across
this ugliness. I had a Java-language processor called Lark - the world's
first - and when XML went case-sensitive, I got a factor of three
performance improvement, it was all being spent in toLowerCase(). -Tim
> -----Original Message----- > From: dcdev-bounces@dcportal.net [mailto:dcdev-bounces@dcportal.net] On > Behalf Of Jacek Sieka > Sent: Wednesday, January 05, 2005 10:48 AM > To: 'Direct Connect developers' > Subject: RE: SV: [dcdev] Anyone still alive? >> > I recently brought case-insensitivity on the development hub, > > and it's a > > noticeable issue. According to Tim Bray at > > http://dotnetjunkies.com/WebLog/sriram/archive/2004/11/18/3270 > > 7.aspx#32794, > > I'm sure hub authors will love that when they were (only partially > > tongue-in-cheek) suggesting $ClientID sorts of mechanisms to save CPU > > time because parsing the tag is just so horribly slow. I'm mostly > > indifferent with mild bias against mandatory case-insensitivity for > > unrelated reasons, but best to know this beforehand.> Indeed. Case sensitive it is then...
Hey there, I think there needs to be a standard. Either case sensitivity or not, and I prefer not. When are case insensitivity matches taking place? Only on login to check user name? In that case, It's not like every command needs these checks, and I'm not sure why this would be heavy for a hub. I kind'a don't like the idea of forcing nick names into a specific case. We already have that when logging into unix machines and in many other computer domains for usernames... Personally I don't like it, but it's not the end of the world.
> > It drastically exceeds a 10Mbit/s connection's capacity, and > > that of a > > 100Mbit/s connection less drastically, even on a slow machine, and in > > the former case by an order of magnitude even when not using > > 100% CPU as > > that test allowed. However, I fully expect that should encryption > > appear, 100Mbit/s users might disable it as I gather they reasonably > > commonly disable zlib compression.> The problem is not the client, the problem is obviously the hub...
Indeed the hub, but I've had another thing in mind. Since some networks actually take action on simple things such as dc filenames, and some networks don't, couldn't it be that some users would need to _require_ encrypted traffic? Today if a 100mbit person disabled zlib, they use extra bandwidth from a 0.5mbit person, so maybe ones gain, but the others pain. When it comes to encryption I think that if it is to be implemented, the 'option of requiring it' should be there... In the cases were some has disabled it, and some requires it, they can't send files in between them, but is that a problem? I think not, it's equally bad for both in the end so...
> > > > I'm unsure as to the nicest way of avoiding ever revealing > > the presence > > of a DC connection by encrypting from the very first packet of a > > connection, but I would strongly support that. > > http://l7-filter.sourceforge.net/technicaldetails and > > http://l7-filter.sourceforge.net/protocols provide insight > > into what the > > programs you mention can detect, and thereby means of defeating them. > > I'm fairly ignorant of the capabilities and limits of the commercial > > versions.> Well, as I see it, we simply create an ssl tunnel which is(?) > indistinguishable from a http ssl tunnel (apart from the fact that the > connections stays up for a long time...). Ssl in theory might be a bit > overkill with certificates and all, but since speeds seem ok, that's > probably the way to go for hubs that want to offer encryption as well...c- > c encryption is a matter of changing protocol in the ctm, and also there > ssl fits the profile... > Trying to make the plaintext protcol look like something else is wasted > effort imho - at some point we'll be sending data which is unique to our > protocol, and all it then takes is a patch to the network filter to block > it...
Off course the plaintext protocol shouldn't be changed, in a try to make it hard to analyse. Some scrambling in a predefined way could do it though, if necessary. It would at least screw up debugging. Encryption in the hub is a bit more difficult than in c-c. When using normal encryption-styles with key-pairs it could be disastrous. Every client would have its own key-pair, and the hub would need to encrypt every message according to every clients public key... So, there another approach could be necessary. Certificates etc aren't really needed, and if I've understood it right TLS is more or less the 'certificate-less ssl'. Does anyone have a good picture of how IPSec works btw? Isn't that encryption on a level (IP) that makes it totally transparent for socket implementations? Or is it that it's not supported by all hardware out there, such as routers? It seems too good not to have in mind at this state. Opera --
On Wed, 5 Jan 2005, [utf-8] Gustaf Räntilä wrote:
> Does anyone have a good picture of how IPSec works btw? Isn't that > encryption on a level (IP) that makes it totally transparent for socket > implementations? Or is it that it's not supported by all hardware out > there, such as routers? It seems too good not to have in mind at this > state.
IPSec is a) at the IP level, which means OS or IP stack, b) a huge and complicated standard and c) doesn't work well with NAT. Not something you want to mess with here. SSL and TLS seem like the best alternatives. /fnord
--
> I recently brought case-insensitivity on the development hub, > and it's a > noticeable issue. According to Tim Bray at > http://dotnetjunkies.com/WebLog/sriram/archive/2004/11/18/3270 > 7.aspx#32794, > I'm sure hub authors will love that when they were (only partially > tongue-in-cheek) suggesting $ClientID sorts of mechanisms to save CPU > time because parsing the tag is just so horribly slow. I'm mostly > indifferent with mild bias against mandatory case-insensitivity for > unrelated reasons, but best to know this beforehand.
Indeed. Case sensitive it is then...
> > Further: I would like not only for encryption, but for some traffic > shaping-resistance in the form of, in Gargoyle's words, doing SSL [or > TLS] from the outset. It even looks pretty practical speedwise; I had > OpenSSL report to me the 1000s of bytes per second processed > under each > cipher on an Athlon 800MHz, and came up with: > > type 16 bytes 64 bytes 256 bytes 1024 > bytes 8192 > bytes > rc4 63129.39k 71440.23k 73317.55k 74151.25k > 74446.17k > aes-128 cbc 18007.45k 18464.32k 18731.69k 18789.38k > 18814.29k > aes-192 cbc 15695.25k 16054.23k 16252.50k 16294.57k > 16315.73k > aes-256 cbc 14000.18k 14208.81k 14367.32k 14404.27k > 14417.92k > > It drastically exceeds a 10Mbit/s connection's capacity, and > that of a > 100Mbit/s connection less drastically, even on a slow machine, and in > the former case by an order of magnitude even when not using > 100% CPU as > that test allowed. However, I fully expect that should encryption > appear, 100Mbit/s users might disable it as I gather they reasonably > commonly disable zlib compression.
The problem is not the client, the problem is obviously the hub...
> > I'm unsure as to the nicest way of avoiding ever revealing > the presence > of a DC connection by encrypting from the very first packet of a > connection, but I would strongly support that. > http://l7-filter.sourceforge.net/technicaldetails and > http://l7-filter.sourceforge.net/protocols provide insight > into what the > programs you mention can detect, and thereby means of defeating them. > I'm fairly ignorant of the capabilities and limits of the commercial > versions.
Well, as I see it, we simply create an ssl tunnel which is(?) indistinguishable from a http ssl tunnel (apart from the fact that the connections stays up for a long time...). Ssl in theory might be a bit overkill with certificates and all, but since speeds seem ok, that's probably the way to go for hubs that want to offer encryption as well...c-c encryption is a matter of changing protocol in the ctm, and also there ssl fits the profile... Trying to make the plaintext protcol look like something else is wasted effort imho - at some point we'll be sending data which is unique to our protocol, and all it then takes is a patch to the network filter to block it... /J --
> Hey there, I think there needs to be a standard. Either case=20 > sensitivity or not, and I prefer not. When are case=20 > insensitivity matches taking place? Only on login to check=20 > user name? In that case, It's not like every command needs=20 > these checks, and I'm not sure why this would be heavy for a=20 > hub. I kind'a don't like the idea of forcing nick names into=20 > a specific case. We already have that when logging into unix=20 > machines and in many other computer domains for usernames...=20 > Personally I don't like it, but it's not the end of the world.
Now hubs have to ensure the nick's unique case-sensitive (this really = means that clients shouldn't be confused/crash (dc++ comes to mind =3D) = by nicks differing only in case). If the hub developer feels like it, = (s)he can still enforce unique case-insensitive nicks (for improved = quality-of-service to the users...). As to speed issues with tolowercase, as long as we only use the first 16 = bits of the unicode chars (which for example windows dows), it's just as = fast as tolowercase'ing a 7-bit ascii string (nearly, disregarding the = doubled memory and lookup table size), so it all boils down to whether = the os supports a good unicode case conversion function, and I believe = most do.=20 That said, I'll go for case sensitive anyway, insensitivity always = brings more work, and the point is to keep the hub rather simple (in a = perfect world, I'd remove the nick constraint, but I realise that people = might not like looking at cid's to tell who's who...) /J=20 --
> > Hey there, I think there needs to be a standard. Either case=20 > > sensitivity or not, and I prefer not. When are case=20 > > insensitivity matches taking place? Only on login to check=20 > > user name? In that case, It's not like every command needs=20 > > these checks, and I'm not sure why this would be heavy for a=20 > > hub. I kind'a don't like the idea of forcing nick names into=20 > > a specific case. We already have that when logging into unix=20 > > machines and in many other computer domains for usernames...=20 > > Personally I don't like it, but it's not the end of the world.> Now hubs have to ensure the nick's unique case-sensitive (this really = > means that clients shouldn't be confused/crash (dc++ comes to mind =3D) = > by nicks differing only in case). If the hub developer feels like it, = > (s)he can still enforce unique case-insensitive nicks (for improved = > quality-of-service to the users...).
Well, this is because there is no Standard that defines case sensitivity usage for the protocol. A clear majority of the hubs use Y(n)Hub which has for a long time been case insensitive. Two users with the same nick (except for their cases) are not allowed, and a user can change nick case and re- login, if I got Yoshi's statement correctly.
> As to speed issues with tolowercase, as long as we only use the first 16 = > bits of the unicode chars (which for example windows dows), it's just as = > fast as tolowercase'ing a 7-bit ascii string (nearly, disregarding the = > doubled memory and lookup table size), so it all boils down to whether = > the os supports a good unicode case conversion function, and I believe = > most do.=20
Yeah, and as I said, case insensitivity matching isn't something that is to be done too often anyway, so for it being to complex for the implementations is a weak argument I think.
> That said, I'll go for case sensitive anyway, insensitivity always = > brings more work, and the point is to keep the hub rather simple (in a = > perfect world, I'd remove the nick constraint, but I realise that people = > might not like looking at cid's to tell who's who...)
And I clearly disagree and believes "user experience" (oh, I hate that expression) should be prioritized higher than a Slightly more complex string routine for nick identification. (Why do I see a lot of "=" and "=20"'s in this mail?) Opera --
On Thursday, January 6, 2005, 21:45:28, Gustaf Räntilä wrote:
> (Why do I see a lot of "=" and "=20"'s in this mail?)
It's Quoted-Printable encoded and missing the Content-Transfer-Encoding: quoted-printable header. -- < Jernej Simoncic ><><><><>< http://deepthought.ena.si/ > Anticipated events never live up to expectations. -- Levy's Second Law of the Disillusionment of the True Liberal --
On Thu, 6 Jan 2005 21:45:28 +0100, Gustaf Räntilä <opera@home.se> wrote:
> Yeah, and as I said, case insensitivity matching isn't something that is to > be done too often anyway, so for it being to complex for the implementations > is a weak argument I think.
Isn't it still valid, if you want to enable people to write hubs with a spare afternoon? Or to keep the code simple and portable, not dependent upon iconv or... whatever? -- Todd Pederzani todd.pederzani@gmail.com --
> > Yeah, and as I said, case insensitivity matching isn't something that is > > to be done too often anyway, so for it being to complex for the > > implementations is a weak argument I think.> > Isn't it still valid, if you want to enable people to write hubs with > a spare afternoon? Or to keep the code simple and portable, not > dependent upon iconv or... whatever?
Yeah, validity indeed but weak as I said, since the result of the protocol shouldn't have much to do with whether an application can be finished within a day or a week. Simplicity and portability... Well yeah, that's probably important *trying to figure out a sound reason for it*... I'd say the dependence on an xml-parser (and why not zlib/bzip2 etc) is reason enough to focus on usability and not simplicity in a sense that makes no sense, unless you're trying to build a hub for a hand held device, like a palm pilot. Read my humble arrogance correctly, I just don't fancy this so over-emphasized issue of super-simplicity. Win32 doesn't need iconv for this issue, even though iconv - as an example - is widely portable. To me, speed issues and Reasonable simplicity are the only really sound ones. And if hubs need to check nick for every mc/pm, and that massively stresses the hubs, and that is reason for case sensitivity, then fine... But then again, last-case-caching shouldn't be too much to ask for, but I dunno, I won't write any hubsoft, I'm just raising the question before protocol gets 'stable' ;) Opera PS / Thanks ender for the explanation / DS --
> Yeah, validity indeed but weak as I said, since the result of=20 > the protocol > shouldn't have much to do with whether an application can be=20 > finished within > a day or a week. Simplicity and portability... Well yeah,=20 > that's probably > important *trying to figure out a sound reason for it*... I'd=20 > say the dependence on an xml-parser (and why not zlib/bzip2=20 > etc) is reason > enough to focus on usability and not simplicity in a sense=20 > that makes no > sense, unless you're trying to build a hub for a hand held=20 > device, like a > palm pilot. Read my humble arrogance correctly, I just don't=20 > fancy this so > over-emphasized issue of super-simplicity. Win32 doesn't need=20 > iconv for > this issue, even though iconv - as an example - is widely portable. > To me, speed issues and Reasonable simplicity are the only=20 > really sound ones. > And if hubs need to check nick for every mc/pm, and that=20 > massively stresses > the hubs, and that is reason for case sensitivity, then=20 > fine... But then > again, last-case-caching shouldn't be too much to ask for,=20 > but I dunno, I > won't write any hubsoft, I'm just raising the question before=20 > protocol gets > 'stable' ;)
Indeed, better to have all things cleared up...in any case, the major = issue with case is that there is no intuitive way of comparing two utf-8 = strings case insensitive - you have to convert them to wide chars. = Without lowercase, the hub doesn't have to know anything about utf-8 or = encodings - it's just shuffling binary data around based on a byte and = the (unescaped) eol marker. /J --
> Indeed, better to have all things cleared up...in any case, the major = > issue with case is that there is no intuitive way of comparing two utf-8 = > strings case insensitive - you have to convert them to wide chars. = > Without lowercase, the hub doesn't have to know anything about utf-8 or = > encodings - it's just shuffling binary data around based on a byte and = > the (unescaped) eol marker.
I don't think I get this completely, looking at e.g. SUP you write that the features should be upper-case only, is this a try to remove all lower-case letters? If so, then fine, clients could send upper-cased nicks whatever user input case was, to move the 'case insensitive match' to the clients, but there's a catch; MSG. Will the chat consist of only upper-cased nicks? That won't look too good, and it would be rather weird to make a utf-8 protocol, but forcing chat nicks to be upper cased. If I've got this right, how do clients know the case of other clients? If nicks are to be case sensitive no matter case, i.e. not to be forced uppered, but 'arne' simply != 'Arne', then why the fuzz about upper cased featurenames for instance? Or was that just a sanitary question to make it look good with a generic case? Then it makes sense. 'Cause case sensitive strcmp in any encoding (such as utf-8) equals memcmp, right? Opera --