ARTEMIS-5745 Unix socket - initial commit

[GChuf]

No description provided.

[jbertram]

What's the essential use-case for this change?

[GChuf]

I would say it's proxying traffic to the socket instead of TCP port for performance and/or security reasons, when using another webserver on the same machine.

[jbertram]

I can see how a Unix socket might perform better than a TCP socket in a relative sense, but what's the use-case for actually needing this kind of performance increase for the embedded web server? I wouldn't expect the embedded web server to be getting a heavy enough load for this to actually make a statistically significant difference.

[GChuf]

I understand your point, however for me it's not about "needing" the extra performance, it's about having the option.

I'm not sure what you mean by statistically significant differences ... I haven't done extensive testing but I would expect this to have a measurable, reproducible impact on website loading times in the range of a couple of percents. This would matter a lot more if the load on the console would be heavier, I completely agree.

I'm sure a number of cases could be made for decoupling messaging traffic from web traffic, and for security.

We're probably looking at this from very different perspectives :)

[jbertram]

...for me it's not about "needing" the extra performance, it's about having the option.

Why would you want to have the option if you don't need the extra performance (assuming the main use-case here really is related to performance)?

I'm not sure what you mean by statistically significant differences...

I mean a meaningful, measurable difference for a real-world use-case. For example, 10% faster average load times for the web console using the default configuration, 20% faster load times for the web console in high-load uses-cases (e.g. lots of addresses & queues), etc. Something like a 2-3% increase probably isn't worth the additional complexity (i.e. technical debt) here.

I'm sure a number of cases could be made for decoupling messaging traffic from web traffic, and for security.

What kind of decoupling did you have in mind? Messaging traffic is handled via Netty by the Core broker which is already separate from web traffic, which is handled by the embedded Jetty instance. Both use independent thread pools. Also, traffic for each goes over different ports and potentially even different network interfaces.

Can you elaborate on the security aspect here?

We're probably looking at this from very different perspectives :)

That's certainly possible. Since you specifically solicited opinions in the description of the Jira I figured I'd jump in.

I'm always trying to better understand how folks use the broker, and I'm not super familiar with Unix domain sockets so I'm keen to learn about the use-case here. Aside from that I'd like to reduce complexity and technical debt where it makes sense. I don't want to fall into premature optimization.

[GChuf]

Why would you want to have the option if you don't need the extra performance (assuming the main use-case here really is related to performance)?

Maybe that was a poor choice of words ... I'll try to explain my reasoning as best as I can. I hope you can understand the tone of the message :)

Need is a strong word. You could say that no-one needs the extra 10% when accessing the web console. Regardless, for the sake of this argument, let's say that I don't "need" this extra 10% at this moment, because the web console seems to be handling things ok. It's nice to have the option! though, because I might need it in the future. I'm expecting my broker to be under heavier loads, and my network to degrade ... etc.
The other point I want to make is, that I'll take 2% improvement any day of the week, if all I need to do is change my proxy configuration from "127.0.0.1" to "/tmp/jetty.sock". From that point of view, this almost seems like a free lunch.

For example, 10% faster average load times for the web console using the default configuration ... Something like a 2-3% increase probably isn't worth the additional complexity (i.e. technical debt) here.

I thought that's what you were getting at. This is also why I wanted some opinions. You're right, this adds additional complexity, and so the question is if it's worth it. Since I'm not an "expert" developer I'm not entirely sure how big the technical debt of adding this feature could potentially be.

I've been trying to benchmark this today but I'm getting wildly different results while benchmarking the same instance - so I don't want to mention any numbers at this moment ... But I intend to post the results here once I'm able to reproduce them.
I agree that this could fall under "premature optimization" if there's no real benefits of using this under heavy load on the broker and/or network ... or the web console.

What kind of decoupling did you have in mind?

I was referring to network traffic, if we assume that all the netty connectors are on the same interface as jetty, but you've already mentioned different interfaces.

Can you elaborate on the security aspect here?

I think most people would mention linux file permissions here, but I don't think I'm suited to defend that point of view.

[jbertram]

But I intend to post the results here once I'm able to reproduce them.

I look forward to that!

Thanks for your investment here. 👍

[GChuf]

This was a bit harder to test that I thought (had some issues with reproducing results and puppeteer memory usage, and so on).

For now I have 3 comparisons. I used the same broker instance for testing this, with this bootstrap.xml config:

       <binding name="artemis" uri="http://0.0.0.0:8162">
           <app name="console" url="console" war="console.war"/>
       </binding>
       <binding name="artemis2" uri="unix:///tmp/jetty.sock">
           <app name="console" url="console" war="console.war"/>
       </binding>

I proxied both addresses via nginx with minimal config.
The broker config wasn't changed in any way.

I tested the "status" console page manually (with chrome dev tools) and the /console/artemis login screen with puppetteer.js:

import puppeteer from "puppeteer";
async function testAverageLoadTime(url, runs = 500) {
  const browser = await puppeteer.launch({
    headless: true,
    args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-cache']
  });
  const page = await browser.newPage();  // Reuse one page
  let totalTime = 0;
  for (let i = 1; i < runs; i++) {
    const start = Date.now();
    await page.goto(url, { waitUntil: "load" });
    const loadTime = Date.now() - start;
    console.log(`Run ${i + 1}: ${loadTime} ms`);
    totalTime += loadTime;
  }
  await page.close();
  await browser.close();
  console.log(`UNIX: Average load time over ${runs} runs: ${totalTime / runs} ms`);
}

The calls via puppeteer to /console/artemis made 10 requests on average. Manual testing the status page had 76 requests.
Puppeteer testing was done on the same machine, while manual testing was done from another PC.
I didn't test with broker under heavy loads yet, but I did try to saturate the loopback interface with iperf3 to see if heavy network traffic would make a difference.
I made 500 runs with puppeteer and 10 manual refreshes on status page.
If you have better ways of testing please let me know.
Otherwise - this comparison really lacks a test where the broker itself is under heavy load (consumers, producers, messages).
Results:

mode	TCP	Unix socket
puppeteer with no load:	125.3ms (min 97, max 193)	124.56ms (min 102, max 167)
puppeteer with saturated lo:	131.33ms (min 97, max 534)	126.96ms (min 102, max 486)
status page with saturated lo:	1,142s (min 1110, max 1170)	1,102s (min 1080, max 1150)

[jbertram]

Do you plan on doing more testing? The current results aren't exactly compelling.