Skip to content

Conversation

@Caiooooo
Copy link

@Caiooooo Caiooooo commented Jan 14, 2025

Fix TLS issue in the origin HTTP client

The origin HTTP client was experiencing problems with TLS, causing failures when making secure requests. This commit updates the TLS configuration to handle h2 connenction correctly.

more specifitily this code was unable to run before.

use std::time::Instant;

use http::{request::Builder, Method};
use monoio_http::common::body::{Body, FixedBody, HttpBody};

#[monoio::main(driver = "uring", enable_timer = true)]
async fn main() {
    let h2_client = monoio_http_client::Builder::new()
        .http2_client()
        .http2_max_concurrent_streams(200)
        .build();

    let base_url = "https://api.binance.com";
    let endpoint = "/api/v3/ticker/price";
    let symbol = "BTCUSDT"; 

    let url = format!("{}?symbol={}", endpoint, symbol);
    let mut latencies = Vec::new();

    for _ in 0..1000 {
        let url = url.clone();

        let request = Builder::new()
            .method(Method::GET)
            .uri(format!("{}{}", base_url, url))
            .version(http::Version::HTTP_2)
            .body(HttpBody::fixed_body(None))
            .unwrap();

        let start_time = Instant::now();

        let resp = h2_client
            .send_request(request)
            .await
            .expect("Failed to send request");

        let (_parts, mut body) = resp.into_parts();
        let mut collected_data = Vec::new();

        while let Some(Ok(data)) = body.next_data().await {
            collected_data.extend_from_slice(&data);
        }
        let s = String::from_utf8(collected_data).unwrap();
        println!("{}", s);

        let duration = start_time.elapsed();
        latencies.push(duration.as_millis() as u64);
    }

    let mut latencies = latencies.clone();

    latencies.sort();

    let p999 = calculate_percentile(&latencies, 99.9);
    let p99 = calculate_percentile(&latencies, 99.0);
    let p90 = calculate_percentile(&latencies, 90.0);
    let p60 = calculate_percentile(&latencies, 60.0);
    let avg = latencies.iter().sum::<u64>() as f64 / latencies.len() as f64;

    println!("P999 latency: {} ms", p999);
    println!("P99 latency: {} ms", p99);
    println!("P90 latency: {} ms", p90);
    println!("P60 latency: {} ms", p60);
    println!("Average latency: {} ms", avg);
}

fn calculate_percentile(latencies: &[u64], percentile: f64) -> u64 {
    let index = (percentile / 100.0 * latencies.len() as f64).ceil() as usize - 1;
    latencies[index]
}

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ Caiooooo
❌ wu_jia_tong


wu_jia_tong seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Member

@ihciah ihciah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's also some detail I don't know. @har23k could you please also have a look at it?

.collect();

encoder.encode(&mut input.clone().into_iter(), &mut buf);
encoder.encode(input.clone().into_iter(), &mut buf);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the borrowed expression implements the required traits
for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrows_for_generic_args
#[deny(clippy::needless_borrows_for_generic_args)] implied by #[deny(warnings)]

a clippy warning, you can change it back.


// (next-state, byte, flags)
pub const DECODE_TABLE: [[(usize, u8, u8); 16]; 256] = [
pub static DECODE_TABLE: [[(usize, u8, u8); 16]; 256] = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we won't change it, put it in .data is better than .bss?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

large array defined as const
for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#large_const_arrays
#[warn(clippy::large_const_arrays)] on by defaultclippy[Click for full compiler diagnostic](rust-analyzer-diagnostics-view:/diagnostic message [0]?0#file:///workspaces/bw/pr/monoio-http/monoio-http/src/h2/hpack/huffman/table.rs)
table.rs(265, 5): make this a static item: static
large array defined as const
for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#large_const_arrays
#[deny(clippy::large_const_arrays)] implied by #[deny(warnings)]clippy[Click for full compiler diagnostic](rust-analyzer-diagnostics-view:/diagnostic message [2]?2#file:///workspaces/bw/pr/monoio-http/monoio-http/src/h2/hpack/huffman/table.rs)
mod.rs(80, 24): the lint level is defined here
table.rs(265, 5): make this a static item: static

this is my editor's warning, you can change it back.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! I did some research and you are right.
With const: it will be replaced and may cause big output;
With static: it not always be compiled to .bss, when it is mutable and uninitialized, it would be put into .bss; otherwise it would be put into .rodata/.data. Here .rodata would be used.

const CONST_ARRAY: [u8; 2] = [1, 2];
static STATIC_ARRAY: [u8; 2] = [1, 2];
static mut STATIC_MUT_ARRAY: [u8; 2] = [1, 2];

I wrote a demo, and objdump the target. I can found STATIC_ARRAY in .rodata, and STATIC_MUT_ARRAY in .data, CONST_ARRAY symbol cannot be found.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your detailed research! 😊

// #[cfg(feature = "rustls")]
// assert_eq!(key.server_name, Some("bytedance.com".try_into().unwrap()));
// #[cfg(all(feature = "native-tls", not(feature = "rustls")))]
// assert_eq!(key.server_name, Some("bytedance.com".into()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this commented out?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please explain it? And if you meant to delete, please not use comment.


pub trait NewTlsStream {
fn new(config: ConnectionConfig) -> Self;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about require From<ConnectionConfig> instead of defining a new trait?

@Caiooooo Caiooooo closed this Feb 10, 2025
@ihciah
Copy link
Member

ihciah commented Feb 10, 2025

Sorry I didn't noticed the PR updating. I'm happy to have people contribute code to this project and would be happy to merge it. Apart from the slow review (which is my fault), is there any other reason why this PR was closed?

@Caiooooo Caiooooo reopened this Feb 10, 2025
.http2_max_concurrent_streams(200)
.build();

let base_url = "https://api.binance.com";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest using some neutral service like httpbin. I'm not sure if there are some policy issues here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @ihciah I’ve addressed the previous feedback and updated the code. Could you please review it again when you have time?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants