Package limn.components

Class VideoView

java.lang.Object
limn.scene.Widget<VideoView>
limn.components.VideoView
public final class VideoView extends Widget<VideoView>
Shows the pictures of a VideoStreamSource: each one handed to a device-side picture and composited as one quad in the 2D paint order, so clipping, opacity, overlays and dialogs apply to a video exactly as they apply to a rectangle.

How a picture reaches the device is the surface's business and not this view's: samples are uploaded and a hardware decoder's handle is bound without a copy, and either way the frame is read and released inside the paint that showed it. This view therefore has no branch for it.

A stream, or a player, and nothing this view opens or closes. setSource(limn.video.VideoStreamSource) shows a stream directly and setPlayer(limn.video.MediaPlayer) shows what a MediaPlayer has already decoded and timed; setting one clears the other, because two things pacing one picture is two clocks disagreeing. Either way the view never calls VideoStreamSource.close() and never calls MediaPlayer.close(): not on replacement, not on detach, not ever. A view that closed what it was given would make setSource(a); setSource(b); setSource(a) impossible and would close a stream a second view is still showing; and a widget that could open one would have to know about decoders, which is the dependency this widget set exists without. What the view does own is its device-side picture and any decoded pictures it is holding, and all of them are released when it is detached. 


 VideoStreamSource stream = Videos.open(file);   // the application's decoder, the application's stream
 VideoView view = new VideoView(stream).setLooping(true);
 // ...or, with a decode thread and a soundtrack behind it:
 MediaPlayer player = new MediaPlayer(stream).setAudio(track, PlayOptions.DEFAULTS);
 VideoView view = new VideoView().setPlayer(player);
 player.start();                                 // the view does not start or stop a player
 // ... and when the application is finished with it:
 player.close();
 stream.close();

Pacing, with a stream. Pictures are shown on a VideoClock driven by the wall clock, so a 30-per-second stream is shown thirty times a second on a display refreshing at any rate, and a held tick costs no repaint at all. The view holds two decoded pictures at most (the one waiting for its moment and the one after it), and that successor is what lets the clock drop: a view that has fallen behind (a slow decode, or a stream whose rate is above the display's) catches up by leaving late pictures unshown, which is what keeps the film at its own speed rather than at whatever the machine could manage. Without the second picture, which is what a source with a single pooled slot leaves it with, nothing is ever dropped and a view that falls behind stays behind.

Pacing, with a player. The player has already decided: it hands over a picture only when that picture's moment has come, having dropped any whose moment passed, and it is what the audio position masters. The view then does nothing but upload and draw. Everything that was the view's (restart(), seek(long, VideoStreamSource.SeekMode), isEnded(), failure(), setLooping(boolean) and clock()) reads or drives the player instead, so there is one answer to each rather than two that can disagree.

A recording made sideways is shown upright. A stream reporting a quarter turn is drawn under that rotation and measures to its displayed size, so a portrait recording asks for a portrait box and letterboxes inside it as a portrait picture. The samples are never turned; the angle is a transform on the quad, which the picture was already going through.

Decoding happens on the UI thread with a stream, at most two pictures per repaint (the one being shown and the one read ahead of it), in the periodic callback that paces the view; a source whose readFrame is slow therefore costs frame time. With a player it happens on that player's decode thread, and the UI thread only takes what is ready.

Points and device pixels. Everything this widget measures, lays out and draws is in logical points. The picture is in device pixels and is the size the stream says it is; VideoSurface.resize(int, int) is a no-op and is never called here. A change of content scale, a window resize and a change of resolution mid-stream are three different events and none of them costs a reallocation on this side: the first two only change the rectangle the composite filters the picture into, and the third is absorbed by the upload, after which the aspect ratio of the letterbox follows the new picture while the measured box keeps following the stream's declared size, so a resolution change re-letterboxes without relaying out the window around it.

What is on screen, in every state a stream can be in. With no stream: nothing at all, and the view asks the layout for no room either. With a stream and no picture yet (including a decoder that has not produced one), the bar colour over the whole box, so the space is visibly reserved rather than showing a hole. Playing: the picture, centred and fitted, over those bars. Ended, whether because looping is off or because the stream cannot be rewound: the last picture, unchanged and costing nothing, until something else is asked of the view. A decode that threw: a short message, and the stream is not read again; failure() has the exception and restart() clears it. No GPU backend at all: a placeholder frame and a message, and the stream is never opened for reading, so a machine with no device decodes nothing.

Reading right to left changes the notice and nothing else. A picture is content: a frame drawn mirrored would reverse the sign on a shop front and turn every hand in it into the other one, so the letterbox, the fit and the rotation are the same arithmetic in both directions. The one thing here that is language and not picture is the message a stalled or unsupported view draws, and it is shaped for this view's direction. Its pill stays where it is: it is centred, and a centre has no side to move to.

Cost while hidden. A view that is not showing (an unselected tab, a scrolled-away row, setVisible(false)) arms no periodic callback, decodes nothing and uploads nothing. The repaint that reveals it starts it again. A player is not stopped by any of that: it keeps decoding until its ring is full and its soundtrack keeps playing, because the player is the application's and a track that stopped when a tab was clicked is not what a player is for. Stop it, or pause it, if that is what the application wants.

Size axis: this widget does not participate. Its content is a decoded picture whose size is the stream's, and the box is the layout's, so no ControlSize step changes what a video looks like. The one thing that is chrome (the "no GPU backend" and "cannot be played" notices) reads the row resolved on this widget like every other component, so a view dropped into an XSMALL panel does not report failure in MEDIUM body type.

To an assistive technology this is one video element over the box the layout gave it, named by the application or by its tooltip and never by this class, described by whichever notice it would draw, and carrying its position in whole seconds whenever the stream has a length; a set on that position reaches the same seek(long, VideoStreamSource.SeekMode) a scrub reaches. Playing and pausing are the transport's there as they are on screen. onAccessibility(limn.accessibility.Accessibility) has the whole of it.

UI thread throughout, like every widget.

  • Constructor Details

  • Method Details

    • setSource

      public VideoView setSource(@Nullable VideoStreamSource newSource)
      Shows source from wherever it currently is, or nothing when null. The previous source is not closed; it is the caller's, and closing it here would break handing the same stream to another view. The previous stream's last picture is not shown again: the box stays empty until the new stream produces one, so two streams never share a frame on screen.

      Re-measures, because the natural size is the stream's.

    • setAutoplay

      public VideoView setAutoplay(boolean newAutoplay)
      Whether a source installed after this call starts playing by itself. False by default: a video starts paused, showing its first picture and holding it, and something has to ask for it to run: setPaused(false), a transport's play button, an application that knows the view is the reason the window was opened.

      Applied when a source or player is installed, so set it before setSource(limn.video.VideoStreamSource) rather than after; on a view that already holds one, setPaused(false) is what starts it.

      Either way a paused video is a still frame, not an empty box: what it shows before it has played is its poster, the first picture unless set otherwise, with a stream and with a MediaPlayer that has not been started alike.

    • isAutoplay

      public boolean isAutoplay()
      Returns:
      whether an installed source starts playing by itself; false by default

    • source

      public @Nullable VideoStreamSource source()
      Returns:
      the stream being shown directly, or null when there is none or a player holds it

    • setPlayer

      public VideoView setPlayer(@Nullable MediaPlayer newPlayer)
      Shows what newPlayer decodes and times, or nothing when null. The previous player is not stopped and not closed, and neither is the stream behind it: both are the caller's. Clears any stream set directly.

      The view does not start the player either; a view that started one would decide when a soundtrack begins, which belongs to whatever opened it. A player that has not been started shows the poster, which for a picture this view cues on the player: decoded, with nothing heard.

      Re-measures, because the natural size is the player's stream's.

    • player

      public @Nullable MediaPlayer player()
      Returns:
      the player driving this view, or null when it is showing a stream directly

    • setPoster

      public VideoView setPoster(Poster newPoster)
      Sets what this view shows before playback has shown a picture of its own: the stream's first picture by default, the picture at a time, an image of the application's, or nothing but the bar colour. Playback begins at the start of the stream whichever it is.

      Shown from when a source or a player is installed until the first setPaused(false); once the video has played, a pause holds the picture it reached. A MediaPlayer that has not started is cued for a picture poster, which decodes it with no sound. An image poster and none read nothing from the stream before playback.

      Parameters:
      newPoster - the poster
      Returns:
      this view

    • poster

      public Poster poster()
      Returns:
      what this view shows before playback; Poster.firstPicture() unless set

    • setFit

      public VideoView setFit(VideoView.Fit newFit)
      How the picture maps into the box when the two aspect ratios differ; VideoView.Fit.CONTAIN by default.

    • fit

      public VideoView.Fit fit()
      Returns:
      how the picture maps into the box

    • setLetterboxColor

      public VideoView setLetterboxColor(Color color)
      The colour behind the picture: the bars beside or above a VideoView.Fit.CONTAIN one, and the whole box whenever there is no picture: before the first arrives, while a stream is being chosen, and when there is no stream at all.

      Opaque black by default, which is what every player uses and is not an arbitrary choice: a bar in any other colour sits directly against the picture's edge and shifts how the tones next to it are read, and black is the one colour that adds no such cast.

      There is no way to turn it off. A view with no background is a hole: the box is reserved and laid out, and whatever happens to be behind it shows through, so an application that has not yet opened a stream displays not "a player waiting" but a gap in its own window that reads as a bug. Being able to paint nothing was worth less than never being able to produce that, so a null or fully transparent colour is refused here rather than becoming a blank rectangle somebody has to explain later. A partly transparent one is allowed: it tints what is behind rather than deleting it.

      Parameters:
      color - the background colour; never null and never fully transparent
      Throws:
      IllegalArgumentException - if color is fully transparent
      NullPointerException - if color is null

    • letterboxColor

      public Color letterboxColor()
      Returns:
      the background colour; never null

    • setPreferredSize

      public VideoView setPreferredSize(float width, float height)
      The size to ask the layout for, in logical points, or -1, -1 (the default) to ask for the stream's own size at one point per pixel. Set it whenever the stream's size is not the size wanted on screen (a 1920×1080 stream asks for 1920×1080 points otherwise) or to reserve the box before a stream has been opened at all.

    • setLooping

      public VideoView setLooping(boolean newLooping)
      Whether reaching the end rewinds and plays again (default false). Ignored by a stream that cannot be rewound, which ends once whatever this says.

    • isLooping

      public boolean isLooping()
      Returns:
      whether the end rewinds rather than stops; the player's answer when one drives this view

    • restart

      public void restart()
      Rewinds the stream to its first picture and re-anchors the pacing, the only repositioning VideoStreamSource defines, and not a seek. Clears an ended or failed state, so it is also how a view recovers from a decode that threw. The picture already on screen stays there until the first one of the new pass is uploaded, so restarting does not flash.

      Does nothing when there is no source.

      Throws:
      UnsupportedOperationException - if the stream cannot be rewound, which VideoStreamSource.canReset() answers in advance

    • seek

      public void seek(long micros, VideoStreamSource.SeekMode mode)
      Moves to micros and carries on from there: the player's seek when one drives this view, and the stream's plus this view's own pacing otherwise. Like restart() it exists because the view holds state the caller cannot reach: the pacing anchor, the picture being held, and the ended and failed flags, none of which a bare VideoStreamSource.seek(long, limn.video.VideoStreamSource.SeekMode) would put right.

      The picture on screen stays there until the first one from the new position is uploaded, so a scrub does not flash. Clears an ended or failed state.

      Cheap enough to call from a control being dragged in VideoStreamSource.SeekMode.KEYFRAME, and VideoStreamSource.SeekMode.EXACT is what lands where the viewer let go. Does nothing when there is no stream.

      Throws:
      UnsupportedOperationException - if the stream cannot be seeked, which canSeek() answers in advance
      IllegalArgumentException - if micros is negative

    • seek

      public void seek(long micros)
      As seek(long, VideoStreamSource.SeekMode) landing exactly.

    • canSeek

      public boolean canSeek()
      Returns:
      whether seek(long, VideoStreamSource.SeekMode) works, which is the stream's answer and false with no stream. A transport control asks this rather than catching an exception under the viewer's finger.

    • positionMicros

      public long positionMicros()
      Returns:
      where the pictures have reached, in microseconds on the stream's own timeline: the player's position when one drives this view and this view's clock otherwise. What a transport control puts its thumb at.

    • setPaused

      public VideoView setPaused(boolean newPaused)
      Freezes the picture where it is, or lets it run again: the player's pause when one drives this view, so the sound freezes with the picture, and this view's own pacing otherwise. Position is kept either way: resuming continues rather than restarting.

      A paused view still decodes one picture and holds it, so resuming shows the next picture immediately rather than after a decode. It reads no further than that: the read-ahead exists to let the clock drop, and a paused clock drops nothing. It does nothing with no stream, and it is not what a stream reaching its end does; that is isEnded().

      A stream carrying no timing at all pauses too. Such a stream is otherwise shown one picture per repaint with no clock consulted, so pausing it has to be answered here rather than by the pacing.

    • isPaused

      public boolean isPaused()
      Returns:
      whether the picture is frozen: the player's answer when one drives this view, so a player paused directly reads as paused here too, and this view's own otherwise

    • isEnded

      public boolean isEnded()
      Returns:
      whether the stream has reported its end and will not be rewound: either because looping is off or because the stream cannot be rewound. The last picture stays on screen; nothing is decoded, uploaded or repainted afterwards.

    • failure

      public @Nullable RuntimeException failure()
      Returns:
      the exception a decode threw, or null. The view then shows a message instead of the stream and stops asking the source for anything; restart() or a new source is what clears it.

    • clock

      public VideoClock clock()
      The clock deciding when each picture is shown: the player's when one drives this view, and this view's own otherwise, so there is one answer rather than two that can disagree. Mutable: install a VideoClock.MasterClock on it to slave the video to an audio position, or pause it to freeze the picture. One clock per view: two views deciding on the same instance would each move the other's timeline.

    • setClock

      public VideoView setClock(VideoClock newClock)
      Replaces the pacing clock, e.g. with one a test drives.
      Throws:
      IllegalStateException - if a player drives this view, whose clock is the one that decides; silently ignoring this would leave a caller holding a clock that paces nothing

    • onAccessibility

      protected void onAccessibility(Accessibility a)
      What a video is to an assistive technology: one Accessible.Role.VIDEO node over the box the layout gave it, carrying the position in whole seconds whenever the stream has a length to measure it against, and described by whichever notice the paint would draw.

      The role is unconditional, and both halves of that are decisions. This class declares no role otherwise, holds no synthetic children and is never focusable, so the transparency rule deletes it and hoists its children — and because it overrides onPaint(limn.graphics.Canvas) it would be named in an application's log as a toolkit class that draws and says nothing, with the recommendation to strike it out, which would take the transport with it. A picture is information rather than a wash, so paintsDecoration() is no more the answer here than it is for an image or a render surface. Nor is the node conditional on holding a stream: a content node that appeared and vanished with setSource(limn.video.VideoStreamSource) would throw away an application's setAccessibleName and its bound caption each time it went, and churn the shape of the tree for a fact nobody asked about. A view with no stream measures 0 × 0 and publishes a 0 × 0 video node, which is what the layout gave it.

      No name is declared, ever. This class holds no string that names it — the two it draws are statuses and not identities — so all three naming routes stay the application's: setAccessibleName, a bound caption, and the tooltip the walk promotes into the name when nothing else has supplied one.

      The notice is a description, and it is read in the paint's own order. onPaint(limn.graphics.Canvas) tests for a backend and returns before it ever reads failure(), so this reads the two the same way round; describing a failure the paint would not draw would be telling a reader about something nobody can see. It is a description rather than a name for the reason a viewport's is: the walk applies an application's name and a bound caption after this hook and unconditionally, so a name written here is only a fallback beneath either, while a status belongs beside whichever of them names the node. The stated cost is a view carrying both a tooltip and an application name, where the description slot is taken by the notice and the tooltip is dropped from the tree until the notice goes. Both branches are read live rather than remembered from the paint, because the tree of a frame is published before that frame's paint; VideoSurfaces.isAvailable() is a volatile read of a static the backend installs before any window exists, so it cannot move underneath a tree being published.

      The position is published in whole seconds, and this widget is why that rule exists. A playing view invalidates on every picture it presents, so the walk runs at the display's rate; an unrounded position would differ on essentially every frame and copy the whole tree, for the length of the film, with nobody touching anything. Rounded, it moves once a second. The display form is the transport's own m:ss and comes from this widget's own memo, not from the bar's clock: that is a different widget, written from a paint and a poll, and the tree of a frame is published before that paint, so quoting it would be a frame stale. What is published is the position alone and never "at / length", because the length is the facet's own maximum and every platform reads that separately — repeating it has it spoken twice.

      There is no facet at all without a length, which is the same gate the transport puts on its scrub bar: VideoStreamSource.durationMicros() answers VideoStreamSource.DURATION_UNKNOWN for a pipe, a live input and a container carrying no duration, and a range with no maximum is not a range. The write bit is canSeek(), said on the facet because the facet's presence is what advertises a set on every platform: seek(long, VideoStreamSource.SeekMode) throws on a stream that cannot be seeked, so a range published without it offers a reader a position it cannot set and throws out of the posted task when the verb arrives anyway.

      No orientation, no busy bit and no play state, each for its own reason. Horizontal and vertical say that a node's value runs along a screen axis, and this value runs in time. BUSY has nothing honest behind it: this widget has no buffering model, and uploaded == false covers both a decoder that is working and a player nobody has started, which are opposite facts. And the state enum carries no bit for playing because no platform does; that fact reaches a reader where it reaches everyone else, at the transport's play button, whose name flips in the frame of the change.

      Nothing is declared about the built-in bar. controls() adds a real MediaControls child that declares its own role, its own orientation and this view as what it controls, and there is nothing the view knows about the bar that the bar does not. Worth saying once: a view showing a transport therefore publishes two settable positions, this node in seconds and the scrub bar in its own thousandths. That is two controls for one thing rather than a contradiction, and it is what a sighted user is given too.

      The box is the walk's, from this widget's own x, y, width and height, and never the picture's: the fitted quad is zero-sized until the first upload and moves under a mid-stream resolution change with no layout behind it, while the box is what the hit test claims, what the layout reserved and what the letterbox is painted over.

      Overrides:
      onAccessibility in class Widget<VideoView>
      Parameters:
      a - the node being described

    • onAccessibilityAction

      protected boolean onAccessibilityAction(Accessible.Action action, Accessible.Argument arg)
      Moves the playhead where an assistive technology asked for it, through the same public seek(long, VideoStreamSource.SeekMode) the transport's scrub bar reaches when a drag settles: the pictures being held are given back, the player or the stream is repositioned, this view's own pacing is re-anchored, and the ended and failed flags are cleared. Exactly, and not by keyframe, because a reader's set is a settled position rather than a drag.

      Each of the three refusals is a way this would otherwise throw or lie. A value that is not a number survives every clamp, so it is turned away before one. A stream with no length was never published as a range, and one that cannot be seeked throws UnsupportedOperationException, so both are re-checked here against exactly what the facet advertised. The clamp is what keeps a negative away from seek, which refuses one. There is no enabled check of its own: the scene has already re-checked this widget, its whole ancestor chain, that it is showing and that it is reachable, and this widget keeps no input path with a guard for one here to mirror.

      A set beyond the end lands on the end rather than being refused, and the end is the stream's true length rather than the whole second the facet publishes as its maximum, so the last fraction of a second of a film is reachable; what is published back is still the whole second below it, which is inside the range this node advertised.

      There is no play or pause verb, and that is a finding rather than a deferral. This class overrides neither the mouse hook nor the key hook and is never focusable, so clicking a video does nothing at all: a toggle here would be an operation a screen-reader user has and nobody else does, and it would reach setPaused(false) and start a soundtrack the application deliberately did not, which is the whole point of setAutoplay(boolean). It would also carry no toggle facet, a video being nothing that is checked, so it would publish a verb with nothing saying which way it goes. Play and pause are the transport's here exactly as they are on screen. Nor is there an increment or a decrement: this class defines no skip interval and no key that steps, so a step verb would publish a grid no gesture on this widget produces, and a reader still positions absolutely through this one.

      Overrides:
      onAccessibilityAction in class Widget<VideoView>
      Parameters:
      action - what was asked
      arg - the position in seconds, for a set
      Returns:
      whether the playhead moved

    • controls

      public MediaControls controls()
      The built-in playback controls, created on the first ask and hidden until setControlsVisible(boolean) shows them. Everything about them is customized on the instance: MediaControls.addLeading(limn.scene.Widget<?>)/MediaControls.addTrailing(limn.scene.Widget<?>) for an application's own volume or subtitle widgets, the position clock, the backdrop — and the direction, which defaults to the media convention (LTR whatever the tree reads) and is re-declared or cleared to inherit through setLayoutDirection.

    • setControlsVisible

      public VideoView setControlsVisible(boolean visible)
      Shows or hides the built-in playback controls over the picture (default hidden).

    • onLayout

      protected void onLayout()
      Description copied from class: Widget
      Containers position children here (measure + Widget.layoutBox(float, float, float, float) per child).
      Overrides:
      onLayout in class Widget<VideoView>

    • onMeasure

      protected Size onMeasure(Constraints constraints)
      The stream's own size at one point per pixel, or the preferred size when one is set, clamped by the constraints, and 0 × 0 with no stream at all, because a view with nothing to show asks for no room. The stream answers its size at open, so this works before a single picture has been decoded; it keeps answering the stream's declared size afterwards, so a resolution change mid-stream re-letterboxes rather than relaying out.

      The constraints win: a box too small, too large or too narrow for what was asked simply clamps, and it is the letterbox that puts the aspect ratio back; measuring does not preserve it, because a parent that stretches one axis has already decided that axis.

      Specified by:
      onMeasure in class Widget<VideoView>

    • onPaint

      protected void onPaint(Canvas canvas)
      Description copied from class: Widget
      Widget's own background/content, in local coordinates.
      Overrides:
      onPaint in class Widget<VideoView>

    • onDetached

      protected void onDetached()
      Description copied from class: Widget
      Called when this widget leaves the scene: detached from the tree, moved into another scene (told this before the Widget.onAttached() of the scene it joins), or still in it when the scene's window closes, which detaches every widget the scene has. What was taken in Widget.onAttached() from something that outlives the window (a listener's place in a list of the process's, a resource of the system's) is given back here.

      Widget.scene() still answers the scene being left, and becomes null once this returns, so releasing something the scene owns needs no field of your own to remember it by. Fires bottom-up, so children run before their parent, and every one of them can still reach the scene.

      Release resources here, but GPU resources must be handed to Scene.disposeLater(limn.graphics.GpuSurface) (disposal needs the owning GL context, which is only current inside a frame). Not called on Widget.setVisible(boolean). Default no-op.

      Overrides:
      onDetached in class Widget<VideoView>