Ember’s web server isn’t really meant for production use. It’s for development purposes, basically to serve your app locally and auto-reload on changes and so on. In summary I would not recommend it for a production deployment. Since the output of an ember build is simply static files you could use pretty much any web server, Apache would work great. I think I’ve seen some Apache configs floating around on these boards. I’m sure you could even write a super basic node.js webserver with very little effort.
TBH though Docker seems like overkill for serving static assets like that (though if you’re serving an API from it too it might make more sense). Deploying to AWS S3 is pretty common (it’s what we do with our apps), cheap, and easy to set up. And of course there are other static file services like it. Just my 2c. Your mileage may vary.
Oh and maybe this is what you were referring to when you referenced a deploy command but if not I highly recommend ember-cli-deploy for setting up deployments.