I see the question's been answered and nearly two years ago. Jumping in for others who might stumble on the thread. I get great results with VideoToolbox as encoder, using either GPU or software to accelerate, depending which machine I am using.
As already mentioned, setting a constant bitrate, and adjusting it upward is key to producing a result that is nearly indistinguishable from a large source file. A constant bitrate is as effective as two-pass encoding for high-quality output, when paired with other key parameters, and is much quicker than two-pass.
May seem counter-intuitive, but a computer running on all threads, full throttle, to encode a video won't give you best results. Several researchers have demonstrated that quality actually goes down if all cpu threads are engaged in encoding; better to use fewer threads and even throttle ffmpeg with a 3rd party app (encoding does not slow down significantly, in my experience). So limit threads on newer multithread desktops and laptops.
Common practice for target bitrates (seen on Netflix, Amazon) vary with resolution, naturally: at least 5,000kbps for 1080p; 3,500 for 720p. For a noticeable improvement in video quality, the encoder bitrate should be set to at least 1.5 times those common practice bitrates: ie, 7,500 for 1080p, 5,250 for 720p. Similarly for 4K GoPros or dash cams.
Often I work with large movie files from my bluray library, and create slimmed-down versions that are 1/3 to 1/2 the size of the original (20G original gives way to a file of 8-10GB with no perceptible loss of quality. Also: framerate. Maintaining the same framerate from source to slimmed-down file is essential, so that parameter is important. Framerate is either 24fps, 25fps, or 30fps for theatrical film, European tv, and North American tv, respectively. (Except in transferring film to a tv screen, 24fps becomes 23.976fps, in most cases.) Of course 60fps is common for GoPro-like cameras, but here 30fps would be a reasonable choice.
It is this control of framerate and bitrate the keeps ffmpeg in check, and gives you predictable, repeatable results. Not an errant, gigantic file that is larger than the one you may have started with.
I work on a Mac, so there may be slight differences on the command line, and here I use VideoToolbox as software encoder, but a typical command reads:
ffmpeg -loglevel error -stats -i source.video -map 0:0 -filter:v fps\=24000/1001 -c:v h264_videotoolbox -b:v 8500k -profile 3 -level 41 -coder cabac -threads 4 -allow_sw:v 1 -map 0:1 -c:a:0 copy -disposition:a:0 default -map 0:6 -c:s:0 copy -disposition:s:0 0 -metadata:g title\=“If you want file title in the metadata, goes here” -default_mode passthrough ‘outfile.mkv’
-loglevel error (to troubleshoot errors)
-stats (provides progess status in terminal window)
-i infile (source video to transcode)
-map 0:0 (specify each stream in the original to map to output)
-filter:v fps\=24000/1001 (framerate of 23.976, like source)
-c:v h264_videotoolbox (encoder)
-b:v (set bitrate, here I chose 8500k)
-profile 3 -level 41 (h264 profile high, level 4.1)
-coder cabac (cabac coder chosen)
-threads 4 (limit of 4 cpu threads, of 8 on this laptop)
-allow_sw:v 1 (using VideoToolbox software encoding for accleration; GPU is not enabled)
-map 0:1 -c:a:0 copy -disposition:a:0 default (copies audio stream over, unchanged, as default audio)
-map 0:6 -c:s:0 copy -disposition:s:0 0 (copies subtitle stream over, not as default ... ie, will play subtitles automatically)
-metadata:g (global metadata, you can reflect filename in metadata)
-default_mode passthrough (allow audio w/o further processing)
outfile (NOTE: no dash precedes filename/path. Chose mkv format to
hold my multiple streams; mp4 or other formats work just fine ... as
long as contents are appropriate for format.)