While it’s not possible to play mkv files directly on the xbox360, you only need to do a few things to remux the video and audio streams into the mp4 format. This means that no re-encoding is done with the video, therefore there is no loss in video quality.

At the time of writing, the xbox360 doesn’t support anything other than 2 channel AAC audio for mp4 streams, so you’ll have to put up with that. It does work well though.

1. Gather video information
mkvinfo movie.mkv
Look for things like the following:

|+ Segment tracks
| + A track
| + Track number: 1
| + Track UID: 1
| + Track type: video
- snip -
| + Default duration: 41.708ms (23.976 fps for a video track)

This shows us track 1 is the video stream. Remember that, it’ll come in handy later. Also, make a note of the fps, if you get the FPS wrong later you’ll have a/v sync issues. Generally track 1 is always the video stream and track 2 is always the main audio stream:


| + A track
| + Track number: 2
| + Track UID: 445320639
| + Track type: audio
- snip -
| + Name: Main audio DTS 1536
| + Audio track
| + Sampling frequency: 48000.000000
| + Channels: 6


Make a note of whether you’re dealing with a DTS stream (as in my example) or an AC3 stream. There is a newer format ACC3 or something..I don’t know much about that.

2. Extract the video and audio streams from the mkv
Remember, mkv and mp4 are just like avi (in some ways.) They are all container formats, not the actual video itself. Now we extract the audio and video streams we want. If you like you could choose different audio and video streams you discovered using mkvinfo.

Extract the video:

mkvextract tracks movie.mkv 1:video.h264


Next we have to do something quite strange in order to make our h.264 file compatible with the xbox360. Run hexedit video.h264 and change the sequence: “67 64 00 33” to “67 64 00 29”.

Now we have our video file ready for muxing, we need to deal with the audio stream:

mplayer movie.mkv -novideo -ao pcm:fast:file=audiodump.wav -channels 2

This gets the audio from the mkv file and saves it as audiodump.wav ready for re encoding using nero’s AACenc program (http://www.nero.com/eng/down-ndaudio.php):

3. Encode the audio dump to AAC format
Now we encode the wav file to AAC:

neroAacEnc -lc -ignorelength -q 0.50 -if audiodump.wav -of audio.m4a

Both the audio and video streams are now ready to be muxed (put together) by MP4Box:

4. Mux the files

Remember to use the correct FPS value we obtained using mkvinfo from earlier!

MP4Box -new output.mp4 -add video.h264 -add audio.m4a -fps 23.976

output.mp4 is now ready to watch!!

Footnotes:
1. I didn’t have any luck getting windows media player 11 to stream the mp4’s to my xbox, I got the all to common ‘format not supported’ thing on the xbox even though they played just fine on WMP11.

2. You’ll want to split your original MKV into just under 4GB chunks if you’re putting the files on anything other than DVD5/DVD9. Do this:
mkvmerge -o output_part.mkv --split 3900M movie.mkv

*note – we leave it just under 4000M for two reasons, firstly fat32 can’t store eactly 4GB, secondly I’m not sure if it’s GiB or GB…!

3. Special thanks to Mike from Linux Love. For some tips that actually worked!

4. I’m not sure if there’s an artificial limit on the file size of mp4s that can be played back, but I will know by this afternoon!

Update:
It appears as though the xbox360 doesn’t like any media files > 4GB, a bit of a shame really. Hopefully the next dashboard update will fix these issues. Until then, looks like we’ll have to put up with split files.