Package limn.components

Class VideoView

java.lang.Object
limn.scene.Widget
limn.components.VideoView
public class VideoView extends Widget
Shows the pictures of a VideoStreamSource: each one handed to a device-side picture and composited as one quad in the 2D paint order, so clipping, opacity, overlays and dialogs apply to a video exactly as they apply to a rectangle.

How a picture reaches the device is the surface's business and not this view's: samples are uploaded and a hardware decoder's handle is bound without a copy, and either way the frame is read and released inside the paint that showed it. This view therefore has no branch for it.

A stream, or a player, and nothing this view opens or closes. setSource(limn.video.VideoStreamSource) shows a stream directly and setPlayer(limn.video.MediaPlayer) shows what a MediaPlayer has already decoded and timed; setting one clears the other, because two things pacing one picture is two clocks disagreeing. Either way the view never calls VideoStreamSource.close() and never calls MediaPlayer.close(): not on replacement, not on detach, not ever. A view that closed what it was given would make setSource(a); setSource(b); setSource(a) impossible and would close a stream a second view is still showing; and a widget that could open one would have to know about decoders, which is the dependency this widget set exists without. What the view does own is its device-side picture and any decoded pictures it is holding, and all of them are released when it is detached. 


 VideoStreamSource stream = Videos.open(file);   // the application's decoder, the application's stream
 VideoView view = new VideoView(stream).setLooping(true);
 // ...or, with a decode thread and a soundtrack behind it:
 MediaPlayer player = new MediaPlayer(stream).setAudio(track, PlayOptions.DEFAULTS);
 VideoView view = new VideoView().setPlayer(player);
 player.start();                                 // the view does not start or stop a player
 // ... and when the application is finished with it:
 player.close();
 stream.close();

Pacing, with a stream. Pictures are shown on a VideoClock driven by the wall clock, so a 30-per-second stream is shown thirty times a second on a display refreshing at any rate, and a held tick costs no repaint at all. The view holds two decoded pictures at most (the one waiting for its moment and the one after it), and that successor is what lets the clock drop: a view that has fallen behind (a slow decode, or a stream whose rate is above the display's) catches up by leaving late pictures unshown, which is what keeps the film at its own speed rather than at whatever the machine could manage. Without the second picture, which is what a source with a single pooled slot leaves it with, nothing is ever dropped and a view that falls behind stays behind.

Pacing, with a player. The player has already decided: it hands over a picture only when that picture's moment has come, having dropped any whose moment passed, and it is what the audio position masters. The view then does nothing but upload and draw. Everything that was the view's (restart(), seek(long, VideoStreamSource.SeekMode), isEnded(), failure(), setLooping(boolean) and clock()) reads or drives the player instead, so there is one answer to each rather than two that can disagree.

A recording made sideways is shown upright. A stream reporting a quarter turn is drawn under that rotation and measures to its displayed size, so a portrait recording asks for a portrait box and letterboxes inside it as a portrait picture. The samples are never turned; the angle is a transform on the quad, which the picture was already going through.

Decoding happens on the UI thread with a stream, at most two pictures per repaint (the one being shown and the one read ahead of it), in the periodic callback that paces the view; a source whose readFrame is slow therefore costs frame time. With a player it happens on that player's decode thread, and the UI thread only takes what is ready.

Points and device pixels. Everything this widget measures, lays out and draws is in logical points. The picture is in device pixels and is the size the stream says it is; VideoSurface.resize(int, int) is a no-op and is never called here. A change of content scale, a window resize and a change of resolution mid-stream are three different events and none of them costs a reallocation on this side: the first two only change the rectangle the composite filters the picture into, and the third is absorbed by the upload, after which the aspect ratio of the letterbox follows the new picture while the measured box keeps following the stream's declared size, so a resolution change re-letterboxes without relaying out the window around it.

What is on screen, in every state a stream can be in. With no stream: nothing at all, and the view asks the layout for no room either. With a stream and no picture yet (including a decoder that has not produced one), the bar colour over the whole box, so the space is visibly reserved rather than showing a hole. Playing: the picture, centred and fitted, over those bars. Ended, whether because looping is off or because the stream cannot be rewound: the last picture, unchanged and costing nothing, until something else is asked of the view. A decode that threw: a short message, and the stream is not read again; failure() has the exception and restart() clears it. No GPU backend at all: a placeholder frame and a message, and the stream is never opened for reading, so a machine with no device decodes nothing.

Reading right to left changes the notice and nothing else. A picture is content: a frame drawn mirrored would reverse the sign on a shop front and turn every hand in it into the other one, so the letterbox, the fit and the rotation are the same arithmetic in both directions. The one thing here that is language and not picture is the message a stalled or unsupported view draws, and it is shaped for this view's direction. Its pill stays where it is: it is centred, and a centre has no side to move to.

Cost while hidden. A view that is not showing (an unselected tab, a scrolled-away row, setVisible(false)) arms no periodic callback, decodes nothing and uploads nothing. The repaint that reveals it starts it again. A player is not stopped by any of that: it keeps decoding until its ring is full and its soundtrack keeps playing, because the player is the application's and a track that stopped when a tab was clicked is not what a player is for. Stop it, or pause it, if that is what the application wants.

Size axis: this widget does not participate. Its content is a decoded picture whose size is the stream's, and the box is the layout's, so no ControlSize step changes what a video looks like. The one thing that is chrome (the "no GPU backend" and "cannot be played" notices) reads the row resolved on this widget like every other component, so a view dropped into an XSMALL panel does not report failure in MEDIUM body type.

UI thread throughout, like every widget.

  • Constructor Details

  • Method Details

    • setSource

      public VideoView setSource(VideoStreamSource newSource)
      Shows source from wherever it currently is, or nothing when null. The previous source is not closed; it is the caller's, and closing it here would break handing the same stream to another view. The previous stream's last picture is not shown again: the box stays empty until the new stream produces one, so two streams never share a frame on screen.

      Re-measures, because the natural size is the stream's.

    • setAutoplay

      public VideoView setAutoplay(boolean newAutoplay)
      Whether a source installed after this call starts playing by itself. False by default: a video starts paused, showing its first picture and holding it, and something has to ask for it to run: setPaused(false), a transport's play button, an application that knows the view is the reason the window was opened.

      Applied when a source or player is installed, so set it before setSource(limn.video.VideoStreamSource) rather than after; on a view that already holds one, setPaused(false) is what starts it.

      A stream shows its first picture either way: a paused video is a still frame, not an empty box. A MediaPlayer that has not been started has decoded nothing and therefore shows nothing, which is that class's contract and not this one's.

    • isAutoplay

      public boolean isAutoplay()
      Returns:
      whether an installed source starts playing by itself; false by default

    • source

      public VideoStreamSource source()
      Returns:
      the stream being shown directly, or null when there is none or a player holds it

    • setPlayer

      public VideoView setPlayer(MediaPlayer newPlayer)
      Shows what newPlayer decodes and times, or nothing when null. The previous player is not stopped and not closed, and neither is the stream behind it: both are the caller's. Clears any stream set directly.

      The view does not start the player either; a view that started one would decide when a soundtrack begins, which belongs to whatever opened it. A player that has not been started shows the bar colour and nothing else.

      Re-measures, because the natural size is the player's stream's.

    • player

      public MediaPlayer player()
      Returns:
      the player driving this view, or null when it is showing a stream directly

    • setFit

      public VideoView setFit(VideoView.Fit newFit)
      How the picture maps into the box when the two aspect ratios differ; VideoView.Fit.CONTAIN by default.

    • fit

      public VideoView.Fit fit()
      Returns:
      how the picture maps into the box

    • setLetterboxColor

      public VideoView setLetterboxColor(Color color)
      The colour behind the picture: the bars beside or above a VideoView.Fit.CONTAIN one, and the whole box whenever there is no picture: before the first arrives, while a stream is being chosen, and when there is no stream at all.

      Opaque black by default, which is what every player uses and is not an arbitrary choice: a bar in any other colour sits directly against the picture's edge and shifts how the tones next to it are read, and black is the one colour that adds no such cast.

      There is no way to turn it off. A view with no background is a hole: the box is reserved and laid out, and whatever happens to be behind it shows through, so an application that has not yet opened a stream displays not "a player waiting" but a gap in its own window that reads as a bug. Being able to paint nothing was worth less than never being able to produce that, so a null or fully transparent colour is refused here rather than becoming a blank rectangle somebody has to explain later. A partly transparent one is allowed: it tints what is behind rather than deleting it.

      Parameters:
      color - the background colour; never null and never fully transparent
      Throws:
      IllegalArgumentException - if color is fully transparent
      NullPointerException - if color is null

    • letterboxColor

      public Color letterboxColor()
      Returns:
      the background colour; never null

    • setPreferredSize

      public VideoView setPreferredSize(float width, float height)
      The size to ask the layout for, in logical points, or -1, -1 (the default) to ask for the stream's own size at one point per pixel. Set it whenever the stream's size is not the size wanted on screen (a 1920×1080 stream asks for 1920×1080 points otherwise) or to reserve the box before a stream has been opened at all.

    • setLooping

      public VideoView setLooping(boolean newLooping)
      Whether reaching the end rewinds and plays again (default false). Ignored by a stream that cannot be rewound, which ends once whatever this says.

    • isLooping

      public boolean isLooping()
      Returns:
      whether the end rewinds rather than stops; the player's answer when one drives this view

    • restart

      public void restart()
      Rewinds the stream to its first picture and re-anchors the pacing, the only repositioning VideoStreamSource defines, and not a seek. Clears an ended or failed state, so it is also how a view recovers from a decode that threw. The picture already on screen stays there until the first one of the new pass is uploaded, so restarting does not flash.

      Does nothing when there is no source.

      Throws:
      UnsupportedOperationException - if the stream cannot be rewound, which VideoStreamSource.canReset() answers in advance

    • seek

      public void seek(long micros, VideoStreamSource.SeekMode mode)
      Moves to micros and carries on from there: the player's seek when one drives this view, and the stream's plus this view's own pacing otherwise. Like restart() it exists because the view holds state the caller cannot reach: the pacing anchor, the picture being held, and the ended and failed flags, none of which a bare VideoStreamSource.seek(long, limn.video.VideoStreamSource.SeekMode) would put right.

      The picture on screen stays there until the first one from the new position is uploaded, so a scrub does not flash. Clears an ended or failed state.

      Cheap enough to call from a control being dragged in VideoStreamSource.SeekMode.KEYFRAME, and VideoStreamSource.SeekMode.EXACT is what lands where the viewer let go. Does nothing when there is no stream.

      Throws:
      UnsupportedOperationException - if the stream cannot be seeked, which canSeek() answers in advance
      IllegalArgumentException - if micros is negative

    • seek

      public void seek(long micros)
      As seek(long, VideoStreamSource.SeekMode) landing exactly.

    • canSeek

      public boolean canSeek()
      Returns:
      whether seek(long, VideoStreamSource.SeekMode) works, which is the stream's answer and false with no stream. A transport control asks this rather than catching an exception under the viewer's finger.

    • positionMicros

      public long positionMicros()
      Returns:
      where the pictures have reached, in microseconds on the stream's own timeline: the player's position when one drives this view and this view's clock otherwise. What a transport control puts its thumb at.

    • setPaused

      public VideoView setPaused(boolean newPaused)
      Freezes the picture where it is, or lets it run again: the player's pause when one drives this view, so the sound freezes with the picture, and this view's own pacing otherwise. Position is kept either way: resuming continues rather than restarting.

      A paused view still decodes one picture and holds it, so resuming shows the next picture immediately rather than after a decode. It reads no further than that: the read-ahead exists to let the clock drop, and a paused clock drops nothing. It does nothing with no stream, and it is not what a stream reaching its end does; that is isEnded().

      A stream carrying no timing at all pauses too. Such a stream is otherwise shown one picture per repaint with no clock consulted, so pausing it has to be answered here rather than by the pacing.

    • isPaused

      public boolean isPaused()
      Returns:
      whether the picture is frozen: the player's answer when one drives this view, so a player paused directly reads as paused here too, and this view's own otherwise

    • isEnded

      public boolean isEnded()
      Returns:
      whether the stream has reported its end and will not be rewound: either because looping is off or because the stream cannot be rewound. The last picture stays on screen; nothing is decoded, uploaded or repainted afterwards.

    • failure

      public RuntimeException failure()
      Returns:
      the exception a decode threw, or null. The view then shows a message instead of the stream and stops asking the source for anything; restart() or a new source is what clears it.

    • clock

      public VideoClock clock()
      The clock deciding when each picture is shown: the player's when one drives this view, and this view's own otherwise, so there is one answer rather than two that can disagree. Mutable: install a VideoClock.MasterClock on it to slave the video to an audio position, or pause it to freeze the picture. One clock per view: two views deciding on the same instance would each move the other's timeline.

    • setClock

      public VideoView setClock(VideoClock newClock)
      Replaces the pacing clock, e.g. with one a test drives.
      Throws:
      IllegalStateException - if a player drives this view, whose clock is the one that decides; silently ignoring this would leave a caller holding a clock that paces nothing

    • controls

      public MediaControls controls()
      The built-in playback controls, created on the first ask and hidden until setControlsVisible(boolean) shows them. Everything about them is customized on the instance: MediaControls.addLeading(limn.scene.Widget)/MediaControls.addTrailing(limn.scene.Widget) for an application's own volume or subtitle widgets, the position clock, the backdrop — and the direction, which defaults to the media convention (LTR whatever the tree reads) and is re-declared or cleared to inherit through setLayoutDirection.

    • setControlsVisible

      public VideoView setControlsVisible(boolean visible)
      Shows or hides the built-in playback controls over the picture (default hidden).

    • onLayout

      protected void onLayout()
      Description copied from class: Widget
      Containers position children here (measure + Widget.layoutBox(float, float, float, float) per child).
      Overrides:
      onLayout in class Widget

    • onMeasure

      protected Size onMeasure(Constraints constraints)
      The stream's own size at one point per pixel, or the preferred size when one is set, clamped by the constraints, and 0 × 0 with no stream at all, because a view with nothing to show asks for no room. The stream answers its size at open, so this works before a single picture has been decoded; it keeps answering the stream's declared size afterwards, so a resolution change mid-stream re-letterboxes rather than relaying out.

      The constraints win: a box too small, too large or too narrow for what was asked simply clamps, and it is the letterbox that puts the aspect ratio back; measuring does not preserve it, because a parent that stretches one axis has already decided that axis.

      Specified by:
      onMeasure in class Widget

    • onPaint

      protected void onPaint(Canvas canvas)
      Description copied from class: Widget
      Widget's own background/content, in local coordinates.
      Overrides:
      onPaint in class Widget

    • onDetached

      protected void onDetached()
      Description copied from class: Widget
      Called when this widget leaves the scene (detached from the tree).

      Widget.scene() still answers the scene being left, and becomes null once this returns, so releasing something the scene owns needs no field of your own to remember it by. Fires bottom-up, so children run before their parent, and every one of them can still reach the scene.

      Release resources here, but GPU resources must be handed to Scene.disposeLater(limn.graphics.GpuSurface) (disposal needs the owning GL context, which is only current inside a frame). Not called on Widget.setVisible(boolean). Default no-op.

      Overrides:
      onDetached in class Widget