I'll answer this myself. Just so its clear to everyone, you CAN connect to your instances of EC2 even though they are being managed by beanstalk. This is helpful because you get to see where things are located. In this case, I didn't know Apache was being used as the webserver for tomcat and had to search for that, but you can find it here as today:
/etc/httpd
Per making changes once you find info like this:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers.html
If you create a folder called .elasticbeanstalk at the root of your project and then create a file called myapp.config.
Setup Apache:
cp conf/httpd/conf.d/enabledeflate.conf /etc/httpd/conf.d/enabledeflate.conf
Then create enabledeflate.conf with something like this:
SetOutputFilter DEFLATE
# mod_deflate configuration
<IfModule mod_deflate.c>
# Restrict compression to these MIME types
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xml+rss
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE text/javascript
AddOutputFilterByType DEFLATE text/css
<IfModule mod_headers.c>
# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
</IfModule>
</IfModule>
A couple of notes:
You may need to restart apache the first time you deploy this.
Make sure you put .elasticbeanstalk in the root of your war file (or git repo)