Package limn.video

Class MediaPlayer

java.lang.Object
limn.video.MediaPlayer
All Implemented Interfaces:
AutoCloseable
public final class MediaPlayer extends Object implements AutoCloseable
Plays a video stream: a decode thread ahead of a small ring of pictures, a VideoClock timing them, and an optional audio track that becomes the clock's master. What a VideoStreamSource lacks to be watchable (somewhere to decode that is not the thread drawing, somewhere to keep a picture that is not yet due, and something to be in time with) is exactly this. 

 VideoStreamSource stream = Videos.open(file);       // the application's decoder, and its stream
 MediaPlayer player = new MediaPlayer(stream);
 player.setAudio(track, PlayOptions.DEFAULTS.withBus(AudioBus.MUSIC));
 player.start();
 // ... and when the application is finished:
 player.close();
 stream.close();                                     // in this order, and only in this order

Who closes what

The video stream is the caller's, opened by the caller and closed by the caller. This player never closes it. That is the same rule a view follows, for the same reason: a stream may outlive a player, be shown in two places, or be handed to a second player afterwards. What the player does promise is that close() returns only once its decode thread has stopped touching the stream, so closing it after that is safe and closing it before is a decode against a torn down decoder.

The audio track is this player's from setAudio(limn.sound.AudioStreamSource, limn.sound.PlayOptions) onwards, and is closed exactly once: by the audio engine when playback ends or is stopped, or by close() when playback never started. There is no path on which the caller should close it and no path on which it leaks. That asymmetry with video is not a preference: handing a source to an audio engine transfers it by contract, including on every failure path, so a player that also closed it would be closing it twice.

Threads

One decode thread per player, started at start() and joined at close(). Nothing else ends it: a player nobody refers to any more still has its thread, asleep once the stream has ended, and is closed, with a warning that names the omission, when the backend ends. A shared pool was rejected: VideoStreamSource.readFrame() may block for a whole decode and promises no bound, so a pool of any size lets one stalled stream stop the others, and a page of thumbnails that never appear is a failure nobody can see the cause of. The honest cost is stated instead: twelve videos on a page are twelve threads, and the expensive part of that is the twelve concurrent decodes, not the twelve threads. Starting a soundtrack takes one more, for as long as the audio engine takes to accept it, for the same reason: see start().

Pictures

The player holds at most ringCapacity() decoded pictures. It allocates none of them: the pool belongs to the stream that produced them, so the ring holds borrowed references and every one of them is released exactly once: when it is shown and handed on, when the clock drops it, or when the player is stopped or closed.

Full: the decode thread stops reading and waits. It does not block forever on a condition it might miss; it waits with a bound, so a lost signal costs latency rather than a player that never plays again, and every consumed picture signals it awake. Holding pictures the consumer has not asked for would also starve the stream's own pool, which needs a free slot to decode into: the effective read-ahead is therefore the smaller of this ring and the stream's spare slots, and a stream reporting VideoStreamSource.Read.PENDING because everything is held is answered the same way as one that is simply not ready.

Empty: takePicture() returns null and the caller keeps showing what it has. The screen never goes blank for want of a picture, and each dry spell is counted once by underruns().

Because the ring can name the picture behind the candidate, this player can drop, unlike a view holding a single picture. A player that has fallen behind skips pictures whose moment has already passed rather than showing them late.

Sound, and being in time with it

With an audio track that sounds, its position becomes the clock's master through AudioMasterClock and the pictures follow it. With no track at all (which is every stream a pure-Java decoder produces, so it is the common case and not the corner), no master is installed and the clock free-runs on the wall clock at the stream's own rate.

A track that does not sound is treated as no master rather than as an error: a machine with no audio device yields a handle whose position is a constant zero, which this player detects at admission and declines to follow, so the video plays at the right rate rather than not at all.

A master is dropped the moment the pictures can no longer follow it, and the wall clock carries on from the position already reached, without a step. Two things cause it: a handle that stops advancing (a device that went away, a track that ended before the pictures did), which the clock's own stall detection catches; and a handle that moves somewhere the pictures are not, which in practice means a looping track wrapping to its start. Neither is recoverable by holding pictures and waiting, which is what a clock left following them would do.

Seeking

seek(long, VideoStreamSource.SeekMode) moves the pictures, the soundtrack and the timeline together, and is a request rather than a completed operation: the ring is released and the clock is moved before it returns, and the stream is repositioned by the decode thread, which may be inside a whole decode at that moment. It is cheap enough to drive from a control being dragged, provided the caller drags in VideoStreamSource.SeekMode.KEYFRAME and lands in VideoStreamSource.SeekMode.EXACT.

See Also:

  • Field Details

    • DEFAULT_RING_CAPACITY

      public static final int DEFAULT_RING_CAPACITY
      Pictures held ahead of the one being shown, when nothing says otherwise.
      See Also:

  • Constructor Details

    • MediaPlayer

      public MediaPlayer(VideoStreamSource video)
      Parameters:
      video - the stream to play, which stays the caller's to close, after this player is closed and not before
      Throws:
      NullPointerException - if video is null

  • Method Details

    • setAudio

      public MediaPlayer setAudio(@Nullable AudioStreamSource newAudio, PlayOptions options)
      Gives this player the video's audio track, which it owns from here on: the audio engine closes it when playback ends or is stopped, and close() closes it if playback never started. The caller must not close it and must not stream it itself.

      Null removes a track that has not been started yet, closing it.

      Parameters:
      options - gain, bus and priority for the track. Looping is this player's to decide, so PlayOptions.loop() is overridden by setLooping(boolean).
      Throws:
      IllegalStateException - if this player has already been started

    • setRingCapacity

      public MediaPlayer setRingCapacity(int pictures)
      How many decoded pictures may be held ahead of the one being shown, at least 1. Larger rides out a slower decoder and a busier machine; smaller costs the stream's pool fewer held slots and shortens the pause a stop has to drain. Beyond a handful it buys nothing a stream's own pool can supply.
      Throws:
      IllegalArgumentException - if pictures is below 1
      IllegalStateException - if this player has already been started

    • ringCapacity

      public int ringCapacity()
      Returns:
      how many decoded pictures may be held ahead of the one being shown

    • setClock

      public MediaPlayer setClock(VideoClock newClock)
      Replaces the clock that times the pictures, for a test driving a whole stream with no real time passing, or a host with a timeline of its own. A master installed on it is replaced when this player starts an audio track that sounds.
      Throws:
      IllegalStateException - if this player has already been started

    • clock

      public VideoClock clock()
      The clock timing the pictures. Touch it only from the UI thread: it is not thread-safe, and this player reads it from there on every takePicture().

    • setOwnsDecodeThread

      public MediaPlayer setOwnsDecodeThread(boolean owns)
      Whether this player starts a decode thread of its own (the default) or leaves decodeStep() to the caller, for a host that owns its threads, or a test that wants decoding to happen exactly when it says so.
      Throws:
      IllegalStateException - if this player has already been started

    • setLooping

      public MediaPlayer setLooping(boolean newLooping)
      Whether reaching the end rewinds and plays again (default false). Ignored by a stream that cannot be rewound. May be changed while playing.

      With an audio track, the master governs the first pass only. At the first loop the pictures' timeline restarts and the track's does not (or does, at its own length, which is not the video's), and one clock cannot be on two timelines at once. The player therefore drops the master there and paces the remaining passes on the wall clock, which keeps them at the right rate instead of holding until the track catches up or racing to catch it. Aligning a loop across both tracks needs a seek, which this does not have.

    • isLooping

      public boolean isLooping()
      Returns:
      whether the end rewinds rather than stops

    • start

      public void start()
      Starts decoding, starts the audio track if there is one, and begins timing. Resumes instead when paused, and does nothing at all when already playing, ended or failed; restart() is what rewinds one of those.

      Starting the audio track is where its ownership passes to the engine, so it happens once and only once however many times this is called. It is handed over on a thread of its own (not the application's worker pool, whose jobs would hold it back), since the engine may open the audio device to take it, which can hold a thread for a few hundred milliseconds: this returns at once, state() is MediaPlayer.State.PLAYING from here, and the pictures are held on the first one until the track sounds (or the engine declines it), so picture and sound begin together. audio() answers Playback.NONE until then; a pause, a seek, a stop or a close made meanwhile applies to the track as it arrives, before it is heard: the engine is given it silent, and it takes its gain once those are applied.

      The pictures wait a quarter of a second at most. A device that takes longer to answer (an output waking up, a sound server that hangs) finds them running on the wall clock, and the track, when it arrives, joins them where they are: the pictures and the track are moved there together, which costs the pictures a short hold while the stream finds the spot. A track that cannot be moved there is let go, and the video plays on silently, as one with no track does; so does one that never arrives.

      The device open still holds the engine. While it takes the track, a call the UI thread makes on the sound engine meanwhile waits for the open to end: a click sound through Sounds.play(limn.sound.AudioClip), a gain set from a slider, Sounds.isAvailable(). An application that makes such calls around its first play opens the device beforehand with Sounds.warmUpAsync(), as for its first click sound.

      Throws:
      IllegalStateException - if this player is closed

    • cue

      public void cue(long micros)
      Shows a picture before playing: on a player that has not started, decoding begins at micros with no sound and no clock, and the first picture from there is handed over once through takePicture() while the player stays MediaPlayer.State.IDLE. That picture is a poster. start() then plays from the beginning, seeking back first when the cue was elsewhere.

      A stream that cannot seek is cued where it is, which for one never played is its first picture. A time past the end hands nothing over, on a looping player too. Nor does a picture that cannot be decoded, which leaves the player idle and not failed: start() then plays from the beginning, or, on a stream that cannot seek, fails as the cue did. Does nothing unless this player is idle, and nothing on one already cued.

      Parameters:
      micros - where the picture is, in microseconds on the pictures' own timeline; not negative
      Throws:
      IllegalArgumentException - if micros is negative
      IllegalStateException - if this player is closed

    • isCued

      public boolean isCued()
      Returns:
      whether a cue(long) is under way toward a picture not yet handed over: false once it has been, and false when the stream ended with none to give

    • pause

      public void pause()
      Freezes the picture and the sound, keeping both positions. Decoding continues until the ring is full, so resuming shows the next picture immediately rather than after a decode.

      The clock is told, which is not optional: a paused audio track reports a frozen position, which is indistinguishable from a device that has died, and a clock not told would hand the timeline to the wall clock and run the video straight through the pause.

      Does nothing unless playing.

    • resume

      public void resume()
      Resumes from where pause() froze it. Does nothing unless paused.

    • stop

      public void stop()
      Ends this playback: the decode thread is stopped and joined, every held picture is released, the clock is reset and the sound is stopped. The video stream is left open, wherever it had reached, and is not rewound.

      Terminal for the audio track. Stopping a stream hands it to the engine to close, and a track cannot be re-opened by something that never opened it, so a player started again after this plays silently. pause() is the resumable one. Idempotent, and harmless on a closed player.

    • restart

      public void restart()
      Rewinds to the first picture and plays from there: stop(), then VideoStreamSource.reset(), then start(), which is also how it clears a failed state. Silent afterwards, for the reason stop() gives: the audio track went with the stop, and a video rewound under a track that was not would be showing the first pictures against the wrong sound.
      Throws:
      UnsupportedOperationException - if the stream cannot be rewound, which VideoStreamSource.canReset() answers in advance
      IllegalStateException - if this player is closed

    • seek

      public void seek(long micros, VideoStreamSource.SeekMode mode)
      Moves to micros and carries on from there: the pictures, the soundtrack and the timeline together.

      Returns as soon as the request is placed, which is not the same as the picture having changed. What has happened by the time it returns: every picture this player was holding has been released, the clock has been moved to micros, and the soundtrack has thrown away what it had queued on the device and reports the new position. What has not happened is the decode: the stream is repositioned by this player's decode thread, at the top of its next pass, which is the only thread allowed to touch it and may be inside a whole decode right now. takePicture() keeps answering null until the first picture from there arrives.

      positionMicros() reports micros in between, exactly, rather than creeping forward while the pictures are being found. A transport control reading it therefore shows where it was told to go and not where the buffering has got to.

      While paused this hands over exactly one picture and then holds again, because a viewer dragging a paused video is asking to see where they have landed. The pause is not lifted.

      Clears an ended state (seeking backwards out of the end is how a viewer replays a part) and clears a failure, since a decode that threw at one position says nothing about another. Looping is unaffected, and a seek is not a loop: the pass the pictures are on does not change, so a soundtrack that is still mastering keeps mastering.

      The audio track is repositioned only if it can be. A soundtrack from the same container as the pictures moves with them; a track from somewhere else, or one on an engine that cannot discard what it has queued, keeps playing where it was, and because that leaves it on a timeline the pictures are not on, this player drops it as its master rather than following it somewhere the pictures will never be. canSeekAudio() answers in advance.

      Parameters:
      micros - where to move to, in microseconds on the pictures' own timeline; not negative
      mode - how close to land, and therefore what this costs. VideoStreamSource.SeekMode.KEYFRAME while a control is being dragged and VideoStreamSource.SeekMode.EXACT when it is let go is what makes a scrub bar cheap while it moves and right when it stops.
      Throws:
      UnsupportedOperationException - if the stream cannot be seeked, which VideoStreamSource.canSeek() answers in advance
      IllegalArgumentException - if micros is negative
      IllegalStateException - if this player is closed

    • seek

      public void seek(long micros)
      As seek(long, VideoStreamSource.SeekMode) landing exactly.

    • canSeek

      public boolean canSeek()
      Returns:
      whether seek(long, VideoStreamSource.SeekMode) works: the stream's answer, which is the one that decides whether a transport control is offered at all

    • canSeekAudio

      public boolean canSeekAudio()
      Returns:
      whether a seek also moves the soundtrack. False with no track, with a track no device would take, and with one the engine cannot discard queued audio for, in which case a seek still moves the pictures and stops timing them by a soundtrack that is now somewhere else. UI thread, because it reads the playback handle.

    • close

      public void close()
      Stops everything and releases every picture this player is holding. Blocks until the decode thread has finished the read it is in, which is the whole point of it: only then can the caller close the video stream without tearing a decoder down underneath a decode. A stream that never returns from a read therefore hangs this call, and that is a defect in the stream rather than something to paper over with a timeout that would make the promise a guess.

      The decode thread is not interrupted. VideoStreamSource promises nothing about interruption, and a decoder woken out of a read could leave its input at a position it cannot describe.

      The audio track is closed here if it never reached the engine, one start() was still handing over included; if it reached the engine, or reaches it after this returns, the engine is told to stop it and closes it. The video stream is not closed. Idempotent.

      Specified by:
      close in interface AutoCloseable

    • takePicture

      public @Nullable VideoFrame takePicture()
      The picture whose moment has come, or null to keep showing the current one. The caller owns what it is given and must release it exactly once.

      Returns null while paused, before the first start (but for the one picture a cue hands over), after the end, after a failure, and whenever nothing in the ring is due yet. Pictures whose moment has already passed are dropped and released here rather than handed over late, which a caller holding a single picture could not do for itself.

      The one exception to the pause is a seek: a player seeked while paused hands over the first picture from the new position and then holds again, so that a paused scrub shows where it landed.

      UI thread. Cheap enough for every frame: it reads the master at most once and allocates nothing.

    • decodeStep

      public MediaPlayer.Step decodeStep()
      One pass of the decode loop: at most one picture read from the stream and handed to the ring. Never blocks on the ring and never spins: a full ring and a stream with nothing ready are both MediaPlayer.Step.IDLE, which the caller answers by waiting a moment.

      This player's own decode thread calls it in a loop. A host that owns its threads calls it itself, from anywhere that is not drawing: a read may take a whole decode.

      Returns:
      what this pass did

    • video

      public VideoStreamSource video()
      Returns:
      the stream being played, never null: for a consumer that needs its declared size or frame rate before a picture exists, which every metadata accessor answers from any thread. Still the caller's to close, and only after this player is closed.

    • state

      public MediaPlayer.State state()
      Returns:
      where this player is in its life; readable from any thread

    • isEnded

      public boolean isEnded()
      Returns:
      whether the stream ended and every decoded picture has been handed out. The last one stays on screen; nothing more is decoded or timed.

    • failure

      public @Nullable RuntimeException failure()
      Returns:
      the exception a decode threw, or null. The player stops reading the stream and hands out nothing more; restart() or stop() clears it.

    • hasAudio

      public boolean hasAudio()
      Returns:
      whether an audio track is sounding, or is waiting for start() to sound

    • isFollowingAudio

      public boolean isFollowingAudio()
      Returns:
      whether the pictures are currently timed by the audio track rather than by the wall clock. False before start(), with no track, with a track no device would take, and after a track has stopped advancing or moved off the pictures' timeline, which is what a status line needs to say why a video is pacing the way it is

    • setGain

      public MediaPlayer setGain(float gain)
      Sets the soundtrack's gain, from 0 (silent) to 1 (as recorded), for the track sounding now and for every one this player hands to the engine later: a restart, a re-open. The track is handed over on a worker, and its handle (audio()) exists only once the engine has answered; a gain written on that handle (a mute) reached a track that had already sounded at the options' gain from its arrival to the next write, about a tenth of a second for a control polling its handle. Set here, it is the gain the track arrives with. Overrides PlayOptions.gain() of setAudio(limn.sound.AudioStreamSource, limn.sound.PlayOptions). UI thread only.
      Parameters:
      gain - the gain; values outside 0 to 1 are clamped
      Returns:
      this player

    • gain

      public float gain()
      Returns:
      the soundtrack's gain: what setGain(float) set, or the options' gain until then

    • audio

      public Playback audio()
      Returns:
      the handle to the sounding audio track, or Playback.NONE when there is none: because there was no track, because no device would take it, because playback has been stopped, or because start() is still handing the track to the engine

    • positionMicros

      public long positionMicros()
      Returns:
      the media position in microseconds: the audio track's while it is mastering, and the wall-clock timeline's otherwise

    • decodedFrames

      public long decodedFrames()
      Returns:
      decoded pictures handed to the ring since construction, dropped ones included

    • underruns

      public long underruns()
      Returns:
      how many times a picture was asked for and the ring was empty while playing, counted once per dry spell rather than once per ask, so it reads as "the decoder fell behind this many times" rather than as a frame count

    • bufferedPictures

      public int bufferedPictures()
      Returns:
      decoded pictures held right now, at most ringCapacity()

    • toString

      public String toString()
      Overrides:
      toString in class Object