Awesome idea of utilizing time needed for SSL establishing to prestart a function! Would this work for cold start of additional functions if there was lets say a burst 50 concurrent requests suddenly coming in at same time? Did you maybe test that? Otherwise love your outside the box thinking :)
Correct me if that's not always the case, but the number of instances (scaled up on the FaaS Provider side), doesn't matter since the client only interacts with the FaaS Gateway that behind the scenes handles all the load balancing, scaling etc.
Meaning the approach described here, is only to notify the gateway which function will be utilized so he can pre-initialize them. For spawning additional instances behind the Gateway, different/additional eBPF program logic would be needed. If even eBPF.
It wasn't tested, but if I had to implement it with eBPF:
- I would attach an XDP program (for layer 4 data like TCP latency, requests per second or some TCP header value) or TC program (for layer 7 like specific application data), or even more specific like sched_switch or softirq_entry eBPF tracepoint
- And based on any of this data (whatever data could indicate that additional resources are needed) I would scale the number of resources (pre-initialize them and avoid cold starts)
I hope this gives a rough idea - but as mentioned I would need to try this - but in theory this should definitely work and provide some additional information that FaaS providers can utilize to make better decisions when to scale up or down additional resources.
That's for the sequential requests under the same TCP+TLS tunnel use case of course.
For concurrent requests, I don't see a big issue rather than just the FaaS gateway having to handle "this concurrent/complex logic" for 50 TCP+TLS separate tunnels.
Correct me if I’m wrong or I’m confused (I’m reading your blog in half sleep here, in CST time zone, lol)…
“If there are any failures in the handshakes”, be it TCP or TLS or anything else, wouldn’t there be wastage of resources if the container(s) are already brought up? How are such scenarios handled with the eBPF approach you mentioned?
Haven't really went into a production setup with this, because there are so many other research topics.
But in general, you could have an eBPF Map where the key would be the function ID and value for each of the keys would be 0 (for cold) or 1 (for warm).
Based on this logic you can either triggered to optimization from the eBPF Program
or not.
Thre most likely a better approach but this one would be easy to implement in my case scenario.
Awesome idea of utilizing time needed for SSL establishing to prestart a function! Would this work for cold start of additional functions if there was lets say a burst 50 concurrent requests suddenly coming in at same time? Did you maybe test that? Otherwise love your outside the box thinking :)
Correct me if that's not always the case, but the number of instances (scaled up on the FaaS Provider side), doesn't matter since the client only interacts with the FaaS Gateway that behind the scenes handles all the load balancing, scaling etc.
Meaning the approach described here, is only to notify the gateway which function will be utilized so he can pre-initialize them. For spawning additional instances behind the Gateway, different/additional eBPF program logic would be needed. If even eBPF.
It wasn't tested, but if I had to implement it with eBPF:
- I would attach an XDP program (for layer 4 data like TCP latency, requests per second or some TCP header value) or TC program (for layer 7 like specific application data), or even more specific like sched_switch or softirq_entry eBPF tracepoint
- And based on any of this data (whatever data could indicate that additional resources are needed) I would scale the number of resources (pre-initialize them and avoid cold starts)
I hope this gives a rough idea - but as mentioned I would need to try this - but in theory this should definitely work and provide some additional information that FaaS providers can utilize to make better decisions when to scale up or down additional resources.
That's for the sequential requests under the same TCP+TLS tunnel use case of course.
For concurrent requests, I don't see a big issue rather than just the FaaS gateway having to handle "this concurrent/complex logic" for 50 TCP+TLS separate tunnels.
Very good & interesting topic! Appreciate the share. I always love to see innovations with eBPF!
Would love to see deep dive on “ solution pairs well with Single Packet Authentication (SPA) “, please.
Correct me if I’m wrong or I’m confused (I’m reading your blog in half sleep here, in CST time zone, lol)…
“If there are any failures in the handshakes”, be it TCP or TLS or anything else, wouldn’t there be wastage of resources if the container(s) are already brought up? How are such scenarios handled with the eBPF approach you mentioned?
Haven't really went into a production setup with this, because there are so many other research topics.
But in general, you could have an eBPF Map where the key would be the function ID and value for each of the keys would be 0 (for cold) or 1 (for warm).
Based on this logic you can either triggered to optimization from the eBPF Program
or not.
Thre most likely a better approach but this one would be easy to implement in my case scenario.