Find line break opportunities in iterator of strings

[rkusa]

First and foremost, thank you very much for this library! I really like how you managed to use LineBreak-12.1.0.txt as a direct build source (through your build.rs) 👏

I have two use-cases, where I am looking for a line breaking algorithm. However, in my use-cases, I don't have one single &str that I can provide to linebreaks. Instead, I usually have different types of tree-based data structures. The issue with that is, that I cannot call linebreaks for each node, since this would add an EOT after each node.

I think a good solution for that would be having a method that takes an iterator of strings, instead of one single string. I did a prototypical implementation of that here: rkusa@33f6c6e - and am now wondering whether you'd be interested in a PR bringing this method upstream? It would just require some additional cleanup (reduce redundancy between both methods and most importantly, find a better name for the method ...) - I just wanted to ask first, before I invest the additional time required for the cleanup.

[axelf4]

Yes I could see this being useful, though the applications I am considering do not require it so I am a bit unsure of the design. FYI there is the xi-unicode crate which does a few things differently, but has LineBreakLeafIter that maybe could work for you.
You would want to know in which segment the break is in, right?, so possibly unicode_segmentation::GraphemeCursor could be an inspiration. Note also that char_indices() is not the same as chars().enumerate() since the former gives byte positions.

[rkusa]

You would want to know in which segment the break is in, right?, so possibly unicode_segmentation::GraphemeCursor could be an inspiration. Note also that char_indices() is not the same as chars().enumerate() since the former gives byte positions.

Thanks for pointing that us, I missed the difference between the two while reading the documentation.

Yes I could see this being useful, though the applications I am considering do not require it so I am a bit unsure of the design. FYI there is the xi-unicode crate which does a few things differently, but has LineBreakLeafIter that maybe could work for you.

I can totally understand that it does not always make sense to take something into a library which is not within the author's use-case. Also, the difference between char_indices() and chars().enumerate() make the implementation a bit more difficult and probably less close to what you already have. That is, I'll close this for now and play a bit more with different implementations and maybe come back with the topic at a later point in time (and maybe also re-evaluating xi-unicode for my use-case). Thanks for the response!

Edit: Another approach could be to simply export this part as a standalone function that others can then use as a building block. Though it might be difficult to come up with a good API to keep track of previous states ...

[truchi]

Disgusting +1 bump.

My usecase is (roughly):

fake_html_for_terminal!( // for demonstration purposes, would be user-supplied
    <bold>
        Hello, <red>world</red>!
    </bold>
);

If I use your lib, this will surely work weird:

fake_html_for_terminal!(<red>multi</red><green>color</green>);

It is true simple usecases deserves simple implementations. Yet it is a shame if we cannot reuse your good work in more involved situations!

You have my great respect for taming the Unicode beast.

[nwoods-cimpress]

This is an old thread, but I'm curious what people think about this (very minute) PR:
#11

No, this doesn't take "an iterator of strings", but it is easy to take a set of strings and concoct an iterator that can be fed into linebreaks_iter in my PR