ByteBuffer to CharBuffer

classic Classic list List threaded Threaded
8 messages Options
Jay
Reply | Threaded
Open this post in threaded view
|

ByteBuffer to CharBuffer

Jay
java.nio.ByteBuffer provides a method asCharBuffer() which returns a
"view" of the ByteBuffer as a character buffer. It however does not
take any arguments. And there is no mention in the Javadocs as to how
it converts from bytes to chars.

1. There should be a method ByteBuffer.asCharBuffer(CharSet) and/or
ByteBuffer.asCharBuffer(String charSet) which allows the user to
explicitly specify the character set to be used for conversion.

2. The method ByteBuffer.asCharBuffer() should mention that this
method uses the platform default charset for bytes-to-chars
conversion.

I can provide a patch with these two changes. What do you guys think?
--
Regards
Jay
Reply | Threaded
Open this post in threaded view
|

Re: ByteBuffer to CharBuffer

Pavel Rappo
Hi Jay,

1. I don't know if you've forgotten to attach the patch you mentioned or it has
been stripped by the mail list. In any case, if it's not too big, please inline
it in your next email.

2. A more appropriate place for this discussion would be another list:

    nio-dev at openjdk dot java dot net
   
Thanks,
-Pavel

> On 6 Feb 2017, at 10:57, Jay <[hidden email]> wrote:
>
> java.nio.ByteBuffer provides a method asCharBuffer() which returns a
> "view" of the ByteBuffer as a character buffer. It however does not
> take any arguments. And there is no mention in the Javadocs as to how
> it converts from bytes to chars.
>
> 1. There should be a method ByteBuffer.asCharBuffer(CharSet) and/or
> ByteBuffer.asCharBuffer(String charSet) which allows the user to
> explicitly specify the character set to be used for conversion.
>
> 2. The method ByteBuffer.asCharBuffer() should mention that this
> method uses the platform default charset for bytes-to-chars
> conversion.
>
> I can provide a patch with these two changes. What do you guys think?
> --
> Regards
> Jay

Reply | Threaded
Open this post in threaded view
|

Re: ByteBuffer to CharBuffer

Jonas Konrad
In reply to this post by Jay
 From what I can tell, asCharBuffer behaves exactly like the other
as*Buffer methods, meaning every char in the CharBuffer is represented
by two bytes in the original ByteBuffer (big- or little-endian depending
on ByteOrder of the ByteBuffer). This means bb.asCharBuffer().get()
behaves like bb.getChar(). asCharBuffer does *not* decode using any
Charset (not the platform charset either), it just interprets the bytes
as 16-bit unsigned integers which happen to be the char type in java. I
guess this technically counts as UTF-16? It doesn't do validity checks
either, though.

If you wish to decode a ByteBuffer, you can use the CharsetDecoder API
directly. I do not believe an asCharBuffer-like method which decodes on
the fly is possible generally for all charsets, because decoding is not
necessarily possible in a random-access buffer due to dynamic-width
encoding.

- Jonas Konrad

On 02/06/2017 11:57 AM, Jay wrote:

> java.nio.ByteBuffer provides a method asCharBuffer() which returns a
> "view" of the ByteBuffer as a character buffer. It however does not
> take any arguments. And there is no mention in the Javadocs as to how
> it converts from bytes to chars.
>
> 1. There should be a method ByteBuffer.asCharBuffer(CharSet) and/or
> ByteBuffer.asCharBuffer(String charSet) which allows the user to
> explicitly specify the character set to be used for conversion.
>
> 2. The method ByteBuffer.asCharBuffer() should mention that this
> method uses the platform default charset for bytes-to-chars
> conversion.
>
> I can provide a patch with these two changes. What do you guys think?
>
Reply | Threaded
Open this post in threaded view
|

Re: ByteBuffer to CharBuffer

Brian Burkhalter-2
Looping in nio-dev which, as Pavel noted, is the apt forum for this topic.

Jonas’s response is correct. The concept of “view buffer” is specified by [1]. In particular the statement

"A view buffer is simply another buffer whose content is backed by the byte buffer."

at least to my interpretation signifies that the content of the view buffer is conceptually mapped bit-by-bit to / from the corresponding bits in the byte buffer. The details are of course subject to byte order and the like, and the interpretation of the bits for floating point values is according to IEEE 754. There is however no encoding or decoding applied.

- Brian

[1] http://download.java.net/java/jdk9/docs/api/java/nio/ByteBuffer.html#views

On Feb 6, 2017, at 5:35 AM, Jonas Konrad <[hidden email]> wrote:

> From what I can tell, asCharBuffer behaves exactly like the other as*Buffer methods, meaning every char in the CharBuffer is represented by two bytes in the original ByteBuffer (big- or little-endian depending on ByteOrder of the ByteBuffer). This means bb.asCharBuffer().get() behaves like bb.getChar(). asCharBuffer does *not* decode using any Charset (not the platform charset either), it just interprets the bytes as 16-bit unsigned integers which happen to be the char type in java. I guess this technically counts as UTF-16? It doesn't do validity checks either, though.
>
> If you wish to decode a ByteBuffer, you can use the CharsetDecoder API directly. I do not believe an asCharBuffer-like method which decodes on the fly is possible generally for all charsets, because decoding is not necessarily possible in a random-access buffer due to dynamic-width encoding.
>
> - Jonas Konrad
>
> On 02/06/2017 11:57 AM, Jay wrote:
>> java.nio.ByteBuffer provides a method asCharBuffer() which returns a
>> "view" of the ByteBuffer as a character buffer. It however does not
>> take any arguments. And there is no mention in the Javadocs as to how
>> it converts from bytes to chars.
>>
>> 1. There should be a method ByteBuffer.asCharBuffer(CharSet) and/or
>> ByteBuffer.asCharBuffer(String charSet) which allows the user to
>> explicitly specify the character set to be used for conversion.
>>
>> 2. The method ByteBuffer.asCharBuffer() should mention that this
>> method uses the platform default charset for bytes-to-chars
>> conversion.
>>
>> I can provide a patch with these two changes. What do you guys think?
>>

Jay
Reply | Threaded
Open this post in threaded view
|

Re: ByteBuffer to CharBuffer

Jay
As a view of the underlying contents, I guess it is correct that no
encoding is applied to coerce a ByteBuffer into a CharBuffer. Maybe
the javadoc for ByteBuffer.asCharBuffer() can emphasize this point. It
is a bit confusing that most places in the JDK where bytes are
converted to chars, a Charset is usually involved whereas here we have
a low level operation - coercing bytes into chars.


On Tue, Feb 7, 2017 at 6:17 AM, Brian Burkhalter
<[hidden email]> wrote:

> Looping in nio-dev which, as Pavel noted, is the apt forum for this topic.
>
> Jonas’s response is correct. The concept of “view buffer” is specified by
> [1]. In particular the statement
>
> "A view buffer is simply another buffer whose content is backed by the byte
> buffer."
>
> at least to my interpretation signifies that the content of the view buffer
> is conceptually mapped bit-by-bit to / from the corresponding bits in the
> byte buffer. The details are of course subject to byte order and the like,
> and the interpretation of the bits for floating point values is according to
> IEEE 754. There is however no encoding or decoding applied.
>
> - Brian
>
> [1]
> http://download.java.net/java/jdk9/docs/api/java/nio/ByteBuffer.html#views
>
> On Feb 6, 2017, at 5:35 AM, Jonas Konrad <[hidden email]> wrote:
>
> From what I can tell, asCharBuffer behaves exactly like the other as*Buffer
> methods, meaning every char in the CharBuffer is represented by two bytes in
> the original ByteBuffer (big- or little-endian depending on ByteOrder of the
> ByteBuffer). This means bb.asCharBuffer().get() behaves like bb.getChar().
> asCharBuffer does *not* decode using any Charset (not the platform charset
> either), it just interprets the bytes as 16-bit unsigned integers which
> happen to be the char type in java. I guess this technically counts as
> UTF-16? It doesn't do validity checks either, though.
>
> If you wish to decode a ByteBuffer, you can use the CharsetDecoder API
> directly. I do not believe an asCharBuffer-like method which decodes on the
> fly is possible generally for all charsets, because decoding is not
> necessarily possible in a random-access buffer due to dynamic-width
> encoding.
>
> - Jonas Konrad
>
> On 02/06/2017 11:57 AM, Jay wrote:
>
> java.nio.ByteBuffer provides a method asCharBuffer() which returns a
> "view" of the ByteBuffer as a character buffer. It however does not
> take any arguments. And there is no mention in the Javadocs as to how
> it converts from bytes to chars.
>
> 1. There should be a method ByteBuffer.asCharBuffer(CharSet) and/or
> ByteBuffer.asCharBuffer(String charSet) which allows the user to
> explicitly specify the character set to be used for conversion.
>
> 2. The method ByteBuffer.asCharBuffer() should mention that this
> method uses the platform default charset for bytes-to-chars
> conversion.
>
> I can provide a patch with these two changes. What do you guys think?
>
>



--
Regards
Jay
Reply | Threaded
Open this post in threaded view
|

Soften interface for javax.management.ObjectName.getInstance and friends

Dave Brosius-2
In reply to this post by Jay
Greetings. the method

public static ObjectName getInstance(String domain,
Hashtable<String,String> table)
         throws MalformedObjectNameException {
         return new ObjectName(domain, table);
     }

in javax.management.ObjectName allows for a provided Hashtable to
specify properties for the objectname.

The semantics of an ObjectName don't consider the order of these
properties, however certain tools like jconsole (when used as a name for
a jmx property) does consider the order.

If you wish to create a folder structure to report metrics in jmx, you
need to use this properties map to specify the folder names. JConsole,
then, uses order of iteration to determine the order of the folder
hierarchy.

Suppose you want a folder hierarchy similar to a package name, you may
specify properties like

table.put("a0", "com");
table.put("a1", "acme");
table.put("name", "MyMetric");

in hopes of producing a metric in JConsole in the folder structure,
com/acme/MyMetric.

The problem is of course, that the argument is a Hashtable, not a Map,
and so the items are not ordered at all, yet JConsole uses iteration
order to build the path, so you may get

acme/ao/MyMetric or MyMetric/acme/ao or .....

This means if you really want to have ordered packages, you have to
derive from Hashtable, and override the entrySet() method, including
that set's iterator() to return the values in the order you wish to have
them shown.

That is really janky.

I'm proposing that the interface for getInstance be softened to

public static ObjectName getInstance(String domain,
                                          Map<String,String> table)

as well as

public ObjectName(String domain, Map<String, String> table)

thoughts?


Reply | Threaded
Open this post in threaded view
|

Re: Soften interface for javax.management.ObjectName.getInstance and friends

Mandy Chung
You should send this topic to serviceability-dev which is the mailing list for JMX and other serviceability related issues.

Mandy

> On Feb 23, 2017, at 4:17 PM, Dave Brosius <[hidden email]> wrote:
>
> Greetings. the method
>
> public static ObjectName getInstance(String domain,
> Hashtable<String,String> table)
>        throws MalformedObjectNameException {
>        return new ObjectName(domain, table);
>    }
>
> in javax.management.ObjectName allows for a provided Hashtable to specify properties for the objectname.
>
> The semantics of an ObjectName don't consider the order of these properties, however certain tools like jconsole (when used as a name for a jmx property) does consider the order.
>
> If you wish to create a folder structure to report metrics in jmx, you need to use this properties map to specify the folder names. JConsole, then, uses order of iteration to determine the order of the folder hierarchy.
>
> Suppose you want a folder hierarchy similar to a package name, you may specify properties like
>
> table.put("a0", "com");
> table.put("a1", "acme");
> table.put("name", "MyMetric");
>
> in hopes of producing a metric in JConsole in the folder structure, com/acme/MyMetric.
>
> The problem is of course, that the argument is a Hashtable, not a Map, and so the items are not ordered at all, yet JConsole uses iteration order to build the path, so you may get
>
> acme/ao/MyMetric or MyMetric/acme/ao or .....
>
> This means if you really want to have ordered packages, you have to derive from Hashtable, and override the entrySet() method, including that set's iterator() to return the values in the order you wish to have them shown.
>
> That is really janky.
>
> I'm proposing that the interface for getInstance be softened to
>
> public static ObjectName getInstance(String domain,
>                                         Map<String,String> table)
>
> as well as
>
> public ObjectName(String domain, Map<String, String> table)
>
> thoughts?
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Soften interface for javax.management.ObjectName.getInstance and friends

Daniel Fuchs
In reply to this post by Dave Brosius-2
Hi Dave,

I'm not sure this is quite a good idea because
order doesn't count when comparing ObjectNames.

So the folder analogy would just be a lure:
/a/b/c would be equal to /a/c/b

That said - I can see value in trying to get rid
of the legacy Hashtable - so adding a new method
that takes a Map<String,String> wouldn't necessarily
be a bad thing :-)

A work around for your use case would be to use:

  static ObjectName getInstance(String name)

instead of

  static ObjectName getInstance(String domain
                                Hashtable<String,String> table)

An object name has both a string representation
and a canonical representation.
IIRC we did try to preserve the original string
representation, even if it's not canonical.
It's also preserved in the serial form.

See:
https://docs.oracle.com/javase/8/docs/api/javax/management/ObjectName.html#getKeyPropertyListString--

Hope this helps,

-- daniel


On 24/02/17 00:17, Dave Brosius wrote:

> Greetings. the method
>
> public static ObjectName getInstance(String domain,
> Hashtable<String,String> table)
>         throws MalformedObjectNameException {
>         return new ObjectName(domain, table);
>     }
>
> in javax.management.ObjectName allows for a provided Hashtable to
> specify properties for the objectname.
>
> The semantics of an ObjectName don't consider the order of these
> properties, however certain tools like jconsole (when used as a name for
> a jmx property) does consider the order.
>
> If you wish to create a folder structure to report metrics in jmx, you
> need to use this properties map to specify the folder names. JConsole,
> then, uses order of iteration to determine the order of the folder
> hierarchy.
>
> Suppose you want a folder hierarchy similar to a package name, you may
> specify properties like
>
> table.put("a0", "com");
> table.put("a1", "acme");
> table.put("name", "MyMetric");
>
> in hopes of producing a metric in JConsole in the folder structure,
> com/acme/MyMetric.
>
> The problem is of course, that the argument is a Hashtable, not a Map,
> and so the items are not ordered at all, yet JConsole uses iteration
> order to build the path, so you may get
>
> acme/ao/MyMetric or MyMetric/acme/ao or .....
>
> This means if you really want to have ordered packages, you have to
> derive from Hashtable, and override the entrySet() method, including
> that set's iterator() to return the values in the order you wish to have
> them shown.
>
> That is really janky.
>
> I'm proposing that the interface for getInstance be softened to
>
> public static ObjectName getInstance(String domain,
>                                          Map<String,String> table)
>
> as well as
>
> public ObjectName(String domain, Map<String, String> table)
>
> thoughts?
>