Single threaded ACE recommendations?

[bioshazard]

I am interested in ACE, but wanted to attempt it using a local model on my GPU. This implies one completion at a time in sequence. Because it must be done sequentially, how should I order the layer execution? Top to bottom over and over? I am not worried about speed so much as completeness, but any feedback on sequential execution would be appreciated, thanks!

[thehunmonkgroup]

If you're just worried about load you could put the completions in a queue and pull them out one at a time.

I'm not sure there's a definitive answer on order, ultimately the layers could operate in an async, independent but coordinated fashion.

If you want an order, though, I'd recommend cycling top -> bottom -> top

[bioshazard]

I am not worried about load so much as worried to compromise the alignment with inappropriate ordering given my endpoint simply does not perform more than one completion at a time. Tho with some of these 7B models maybe I can try to load up 2 and load balance. Queue is a great idea.

In any case, your recommendation run top to bottom then back up again is precisely the direction I wanted thank you!!