I was trying to build a Jupyter notebook solution for outlier analysis of video dataset. I wanted to use Video widget for that purpose, but I didn't find in the documentation how to get a current video frame and/or scroll to needed position by calling some widget's method. My problem is very similar (virtually the same) to these unanswered questions one and two.
I managed to implement the idea by saving the video frames to numpy array and employing matplotlib's imshow function, but video playing is very jittering. I used blitting technique to get some extra fps, but it didn't help much, and in comparison, Video widget produces a smoother experience. It looks like Video widget is essentially a wrapper for browser's built-in video player.
Question: How can I get control of widget's playing programmatically so I can synchronize multiple widgets? Could some %%javascript magic help with a IPython.display interaction?
Here below is a Python code snippet just for illustration purposes, to give you an idea of what I want to achieve.
%matplotlib widget
from videoreader import VideoReader # nice pithonic wrapper for video reading with opencv
import numpy as np
import matplotlib.pyplot as plt
from ipywidgets import IntSlider, Play, link, HBox, VBox
# prepare buffered video frames
vr = VideoReader('Big.Buck.Bunny.mp4')
fps = vr.frame_rate
frames = []
for frame in vr[0:300:1]:
frames.append(frame[:,:,::-1]) # brg2rgb
del vr
vi_buff = np.stack(frames, axis=0) # dimensions (T, H, W, C)
# simulate random signal for demonstration purposes
t = np.linspace(0.0, vi_buff.shape[0], num=vi_buff.shape[0]*10)
s = np.sin(2*np.pi*t)*np.random.randn(vi_buff.shape[0]*10)
plt.ioff()
fig = plt.figure(figsize=(11, 8))
ax1 = plt.subplot2grid((6, 6), (0, 0), rowspan=2, colspan=3)
ax2 = plt.subplot2grid((6, 6), (0, 3), colspan=3)
ax3 = plt.subplot2grid((6, 6), (1, 3), colspan=3)
plt.ion()
# initial plots
img = ax1.imshow(vi_buff[0,...])
l0 = ax2.plot(t, s)
l1 = ax3.plot(t, -s)
# initial scene
lo_y, hi_y = ax2.get_ybound()
ax2.set_xbound(lower=-12., upper=2.)
ax3.set_xbound(lower=-12., upper=2.)
def update_plot(change):
val = change.new
img.set_data(vi_buff[val,...])
ax2.axis([val - 12, val + 2, lo_y, hi_y])
ax3.axis([val - 12, val + 2, lo_y, hi_y])
fig.canvas.draw_idle()
player = Play(
value=0, #intial frame index
min=0,
max=vi_buff.shape[0]-1,
step=1,
interval=int(1/round(fps)*1000) #referesh interval in ms
)
fr_slider = IntSlider(
value=0,
min=0,
max=vi_buff.shape[0]-1
)
fr_slider.observe(update_plot, names='value')
link((player,"value"), (fr_slider,"value"))
VBox([HBox([player, fr_slider]), fig.canvas])
I had to learn it the hard way and write my own custom Video widget. Thanks to the authors of ipywidgets module, it was not from the scratch. I also found out that, what I wanted to do, possibly could be done in the easier way with PyViz Panel Video. I am sharing my solution if anyone interested and it could save you the time learning the front-end and Backbone.js.
Caution: it won't run in JupyterLab, you need to switch to Jupyter Classic Mode. Please share a solution in the comments, if you know how to fix: “Javascript Error: require is not defined”
Widget Model on the Python side
from traitlets import Unicode, Float, Bool
from ipywidgets import Video, register
#register
class VideoE(Video):
_view_name = Unicode('VideoEView').tag(sync=True)
_view_module = Unicode('video_enhanced').tag(sync=True)
_view_module_version = Unicode('0.1.1').tag(sync=True)
playing = Bool(False, help="Sync point for play/pause operations").tag(sync=True)
rate = Float(1.0, help="Sync point for changing playback rate").tag(sync=True)
time = Float(0.0, help="Sync point for seek operation").tag(sync=True)
#classmethod
def from_file(cls, filename, **kwargs):
return super(VideoE, cls).from_file(filename, **kwargs)
Widget View on the front-end
%%javascript
require.undef('video_enhanced');
define('video_enhanced', ["#jupyter-widgets/controls"], function(widgets) {
var VideoEView = widgets.VideoView.extend({
events: {
// update Model when event is generated on View side
'pause' : 'onPause',
'play' : 'onPlay',
'ratechange' : 'onRatechange',
'seeked' : 'onSeeked',
},
initialize: function() {
// propagate changes from Model to View
this.listenTo(this.model, 'change:playing', this.onPlayingChanged); // play/pause
this.listenTo(this.model, 'change:rate', this.onRateChanged); // playbackRate
this.listenTo(this.model, 'change:time', this.onTimeChanged); // currentTime
},
// View -> Model
onPause: function() {
this.model.set('playing', false, {silent: true});
this.model.save_changes();
},
// View -> Model
onPlay: function() {
this.model.set('playing', true, {silent: true});
this.model.save_changes();
},
// Model -> View
onPlayingChanged: function() {
if (this.model.get('playing')) {
this.el.play();
} else {
this.el.pause();
}
},
// View -> Model
onRatechange: function() {
this.model.set('rate', this.el.playbackRate, {silent: true});
this.model.save_changes();
},
// Model -> View
onRateChanged: function() {
this.el.playbackRate = this.model.get('rate');
},
// View -> Model
onSeeked: function() {
this.model.set('time', this.el.currentTime, {silent: true});
this.model.save_changes();
},
// Model -> View
onTimeChanged: function() {
this.el.currentTime = this.model.get('time');
},
});
return {
VideoEView : VideoEView,
}
});
And the actual job I needed to get done
from ipywidgets import link, Checkbox, VBox, HBox
class SyncManager():
'''
Syncing videos needs an explicit setting of "time" property or clicking on the progress bar,
since the property is not updated continuously, but on "seeked" event generated
'''
def __init__(self, video1, video2):
self._links = []
self._video1 = video1
self._video2 = video2
self.box = Checkbox(False, description='Synchronize videos')
self.box.observe(self.handle_sync_changed, names=['value'])
def handle_sync_changed(self, value):
if value.new:
for prop in ['playing', 'rate', 'time']:
l = link((self._video1, prop), (self._video2, prop))
self._links.append(l)
else:
for _link in self._links:
_link.unlink()
self._links.clear()
video1 = VideoE.from_file("images/Big.Buck.Bunny.mp4")
video1.autoplay = False
video1.loop = False
video1.width = 480
video2 = VideoE.from_file("images/Big.Buck.Bunny.mp4")
video2.autoplay = False
video2.loop = False
video2.width = 480
sync_m = SyncManager(video1, video2)
VBox([sync_m.box, HBox([video1, video2])])
Related
My aim is to add buttons below my player that jump to specific moments in the video.
There's a demo example that does basically what I want:
https://flowplayer.com/demos/using-the-api — however it is based on the cloud player implementation of Flowplayer, and I'm using the Javascript API to embed my player. The basic relevant script is this:
flowplayer.cloud.then(function () {
var api = flowplayer("#player")
seek3.onclick = function() {
api.fas.seek_to('00:01:01:11');
}
});
An example I found of creating buttons using the Javascript API is as follows:
flowplayer(function(opts, root, api) {
// small jQuery-compatible utility built inside the player
var $ = flowplayer.mq;
// render a custom button when the player is mounted
api.on('mount', function() {
var btn = $('<button>').txt('Test Button').on('click', function() {
api.toggleMute();
});
$('.fp-controls', root).append(btn);
});
});
That works fine with my player. When I try to merge the two approaches, though, I fail. Here is the broken code:
flowplayer(function(opts, root, api) {
var api = flowplayer("#flowplayer");
seek3.onclick = function() {
api.fas.seek_to('00:01:01:11');
}
});
I tried also as alternatives
document.getElementById('seek3').onclick = function()
And
$('#seek1').onclick = function() {
(preceded by the additional line of code borrowed from the working example):
var $ = flowplayer.mq;
but with every variation I keep getting the following error: "Uncaught TypeError: Cannot set property 'onclick' of null".
Any help would be much appreciated. I've found the Flowplayer documentation really difficult to use, since in search results it's often hard to even tell which version of the player is being referenced. I would really like to find some basic info about interacting with the API in addition to solving this particular problem.
For anyone else who might be having difficulties interacting with the API, Flowplayer kindly posted a new demo example -- with similar functionality as in the cloudplayer demo, but this one for the javascript player: https://codepen.io/team/flowplayer/pen/KOzWOK
var api = flowplayer("#player",
{ src : "//edge.flowplayer.org/drive.m3u8"
, autoplay : false
, token : "eyJraWQiOiJqOEF6djg5NlJMMWIiLCJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiJ9.eyJjIjoie1wiYWNsXCI6NixcImlkXCI6XCJqOEF6djg5NlJMMWJcIixcImRvbWFpblwiOltcIm5ncm9rLmlvXCJdfSIsImlzcyI6IkZsb3dwbGF5ZXIifQ.UtJliSh4IcIKs71PVzXWdIrJJ8-1_KRK0hKd7OKp5EJhAdpZStuYbbU4bgDs--C6Gak6OBrb-xQBh6sd4dyrlA"
, muted : true
})
/**
* helper function to seek if possible or
* tell the player to seek when possible
*
* btn {HTMLElement}
* seconds {Number}
*/
function chapter (btn, seconds) {
btn.onclick = function () {
// player is preload=none so we must tell
// the player to seek when it is possible
if (api.readyState == 0) api.setOpts({start_time: seconds})
// player has already started loading content
if (api.readyState > 0) api.currentTime = seconds
// attempt to play content
api.togglePlay(true)
}
}
// bind the buttons to times
chapter(seek1, 20)
chapter(seek2, 45)
chapter(seek3, 60)
My working version includes the following variations on the JS code:
/**
* helper function to seek if possible or
* tell the player to seek when possible
*
* seconds {Number}
*/
function seek_to_cue (seconds) {
//alert('bingo?'); //tft
// player is preload=none so we must tell the player to seek when it is possible
if (api.readyState == 0) {
api.setOpts({start_time: seconds});
}
// player has already started loading content
if (api.readyState > 0) {
api.currentTime = seconds;
}
// attempt to play content
api.togglePlay(true);
}
and in my PHP code which includes the JS player call:
$button_actions .= 'document.getElementById("'.$button_id.'").addEventListener("click", function(){ seek_to_cue('.$start_time_seconds.'); });';
I have angularjs java web-app and on my local computer work perfect , ng-table and everything. I use GoogleChrome .
Problem is, when I put app on another computer , NG-table not show my data . I use also Chrome and same war.
This is code where reload ng-table and I get data in value but they not show. Have someone idea what is problem, I think something with browser becose same war and same code not work same and get identical data.
$scope.$watch(function () { return commonFactory.getList(); }, function (value) {
if( value!=undefined && value[0]!=undefined){
var r = value[0].countNgTable;
}
var initialParams = {
count: r
};
var initialSettings = {
// page size buttons (right set of buttons in demo)
counts: [10,25,50,100,1000],
// determines the pager buttons (left set of buttons in demo)
paginationMaxBlocks: 23,
paginationMinBlocks: 2,
dataset:angular.copy(value)
};
self.tableParams = new ngTableParams(initialParams, initialSettings);
self.tableParams.match=function(t){
commonFactory.setCounter(t)
}
});
I want to save the state where the user left the webView (which is loading a local html file) so that the next time when he enters the same view, he is taken to the same line of text that he was reading the previous time.
I think that could be achieved by saving the scroll value in an integer every time the user swipe the document, and when he enters the view the next time we use the value that was saved previously and scroll the view using a javascript command or "CG" things.
Any idea?
and an objective c solution:
#interface ViewController () <UIWebViewDelegate>
#property (weak, nonatomic) IBOutlet UIWebView *webView;
#end
#implementation ViewController
- (void)viewDidLoad {
[super viewDidLoad];
[self.webView loadRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:#"http://yourwebsite.com"]]];
}
- (void)webViewDidFinishLoad:(UIWebView *)webView {
NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
if ([userDefaults objectForKey:#"contentOffset"]) {
webView.scrollView.contentOffset = CGPointFromString([userDefaults objectForKey:#"contentOffset"]);
}
}
- (void)viewWillDisappear:(BOOL)animated {
[super viewWillDisappear:animated];
NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
[userDefaults setObject:NSStringFromCGPoint(self.webView.scrollView.contentOffset) forKey:#"contentOffset"];
[userDefaults synchronize];
}
From Lyndsey Scott's Answer:
To save and retrieve your scrollview offset from NSUserDefaults, set your UIWebView's delegate and try this:
var viewLaidoutSubviews = false // <-- variable to prevent the viewDidLayoutSubviews code from happening more than once
// Save your content offset when the view disappears
override func viewWillDisappear(animated: Bool) {
var userDefaults = NSUserDefaults.standardUserDefaults()
userDefaults.setValue(NSStringFromCGPoint(myBrowser.scrollView.contentOffset), forKey: "scroll_offset")
userDefaults.synchronize()
viewLaidoutSubviews = false
}
// Retrieve and set your content offset when the view re-appears
// and its subviews are first laid out
override func viewDidLayoutSubviews() {
if (!viewLaidoutSubviews) {
// If a scroll_offset is store in NSUserDefaults, use it
var userDefaults = NSUserDefaults.standardUserDefaults()
var scrollOffset:CGPoint? = CGPointFromString(userDefaults.valueForKey("scroll_offset") as? NSString)
if let position:CGPoint = scrollOffset {
myBrowser.scrollView.delegate = self
myBrowser.scrollView.setContentOffset(position, animated: false)
}
viewLaidoutSubviews = true
}
}
And utilize your UIWebView's webViewDidFinishLoad delegate method to update the scroll offset in the case that the web view didn't finish rendering before the view laid out its subviews:
// Retrieve and set your content offset once the web view has
// finished rendering (in case it hasn't finished rendering
// by the time the view's subviews were laid out)
func webViewDidFinishLoad(webView: UIWebView) {
// If a scroll_offset is store in NSUserDefaults, use it
var userDefaults = NSUserDefaults.standardUserDefaults()
var scrollOffset:CGPoint? = CGPointFromString(userDefaults.valueForKey("scroll_offset") as? NSString)
if let position:CGPoint = scrollOffset {
myBrowser.scrollView.delegate = self
myBrowser.scrollView.setContentOffset(position, animated: true)
}
}
But note, your NSUserDefaults value will also persist between app launches unless you delete it.
I have a number of Scala function calls in my template file, unfortunately they are getting automatically called during template loading for some reason. How can I prevent those from being called?
My intention is they are called only after certain click events. And I would get a huge performance increase during template load (26s vs 3s).
I have a DataLoader Java object, which is being called from template and does the reading of values from database. The relevant parts of DataLoader:
public void SetAllowLoading() {
System.out.println("DataLoader SetAllowLoading > ");
allowLoading = 1;
}
public void SetDisAllowLoading() {
allowLoading = 0;
}
public void debugdebug(String text) {
System.out.println(text);
}
public List<Double> loadAreaLengthData() {
List<Double> areaLengthArray = new ArrayList<Double>();
System.out.println("DataLoader OUT loadAreaLengthData > ");
if (allowLoading > 0) {
System.out.println("DataLoader IN loadAreaLengthData > ");
areaLengthArray.add(Component.getPipeLenghtsAccordingToRelativeFloorAreaMeters(0, 11));
areaLengthArray.add(Component.getPipeLenghtsAccordingToRelativeFloorAreaMeters(11, 21));
areaLengthArray.add(Component.getPipeLenghtsAccordingToRelativeFloorAreaMeters(21, 31));
areaLengthArray.add(Component.getPipeLenghtsAccordingToRelativeFloorAreaMeters(31, 41));
areaLengthArray.add(Component.getPipeLenghtsAccordingToRelativeFloorAreaMeters(41, 51));
}
return areaLengthArray;
}
If loading is not allowed, the loading method doesn't read from database.
Then the necessary parts from template (pipeIndex.scala.html)
$("#info2").on("click", function() {
if($(this).hasClass("selected")) {
$("#info2").html('#Messages("consequence.floor") (m<sup>2</sup>/m)');
#loader.debugdebug("debugFloorAreaDESELECT");
deselectArea();
} else {
$(this).addClass("selected");
$(".chartsarea").slideFadeToggle(function() {
#loader.debugdebug("drawFloorAreaChart()");
var sarea = new Array();
var i = 0;
var number = 0;
#for(s <- loader.loadAreaLengthData) {
number = Math.round(#s);
if (!isNaN(number))
sarea[i] = Math.round(number / 1000);
else
sarea[i] = 0;
i++;
}
var ticksArea = ["0-10", "10-20", "20-30", "30-40", "40-50"];
areaObj = {
// Only animate if we're not using excanvas (not in IE 7 or IE 8)..
animate: !$.jqplot.use_excanvas,
seriesDefaults:{
renderer:$.jqplot.BarRenderer,
pointLabels: { show: true }
},
axes: {
xaxis: {
renderer: $.jqplot.CategoryAxisRenderer,
ticks: ticksArea
}
},
highlighter: { show: false }
}
plot3 = $.jqplot('chartarea', [sarea], areaObj);
$("#info2").focus();
$("#info2").html('#Messages("consequence.hide.floor")');
});
}
return false;
});
function deselectArea() {
$(".chartsarea").slideFadeToggle(function() {
$("#info2").removeClass("selected");
});
}
My question is how that "on" click handler is called automatically at every pageload? How can I prevent the calling during pageload?
I used for showing/hiding functionality the example at jsfiddle: anotherjsfiddle However I used multiple click handlers (here is shown only for info2 element). And I changed that jquery "live" event method to "on", because "live" is deprecated.
If I understand you right you are asking why the debug messages (e.g. #loader.debugdebug("debugFloorAreaDESELECT");) are getting called on each site load and not only on click..?
This is because Play templates are getting rendered on the server side, i.e. as soon as template.render() will get called in the controller. This affects all the the Scala parts in the template (i.e. everything starting with #)
If you would like to debug on the client side you could use JavaScripts console.log()
Although previous answer describes it I'll use a phrase which I repeated several times already:
Play template is ServerSide technology, so it's rendered during Play pass (i.e. if your views will be cached for 1 month you will have 1 month old values in the view), JavaScript is ClientSide. That means - that you can't mix it like this.
In Firefox, at the start of modules/devtools/inspector/inspector-panel.js you see some references to a "walker", shown at the end of this snippet:
...
/**
* Represents an open instance of the Inspector for a tab.
* The inspector controls the highlighter, the breadcrumbs,
* the markup view, and the sidebar (computed view, rule view
* and layout view).
*
* Events:
* - ready
* Fired when the inspector panel is opened for the first time and ready to
* use
* - new-root
* Fired after a new root (navigation to a new page) event was fired by
* the walker, and taken into account by the inspector (after the markup
* view has been reloaded)
* - markuploaded
* Fired when the markup-view frame has loaded
* - layout-change
* Fired when the layout of the inspector changes
* - breadcrumbs-updated
* Fired when the breadcrumb widget updates to a new node
* - layoutview-updated
* Fired when the layoutview (box model) updates to a new node
* - markupmutation
* Fired after markup mutations have been processed by the markup-view
* - computed-view-refreshed
* Fired when the computed rules view updates to a new node
* - computed-view-property-expanded
* Fired when a property is expanded in the computed rules view
* - computed-view-property-collapsed
* Fired when a property is collapsed in the computed rules view
* - rule-view-refreshed
* Fired when the rule view updates to a new node
*/
function InspectorPanel(iframeWindow, toolbox) {
this._toolbox = toolbox;
this._target = toolbox._target;
this.panelDoc = iframeWindow.document;
this.panelWin = iframeWindow;
this.panelWin.inspector = this;
this._inspector = null;
this._onBeforeNavigate = this._onBeforeNavigate.bind(this);
this._target.on("will-navigate", this._onBeforeNavigate);
EventEmitter.decorate(this);
}
exports.InspectorPanel = InspectorPanel;
InspectorPanel.prototype = {
/**
* open is effectively an asynchronous constructor
*/
open: function InspectorPanel_open() {
return this.target.makeRemote().then(() => {
return this._getWalker();
}).then(() => {
return this._getDefaultNodeForSelection();
}).then(defaultSelection => {
return this._deferredOpen(defaultSelection);
}).then(null, console.error);
},
get inspector() {
if (!this._target.form) {
throw new Error("Target.inspector requires an initialized remote actor.");
}
if (!this._inspector) {
this._inspector = InspectorFront(this._target.client, this._target.form);
}
return this._inspector;
},
_deferredOpen: function(defaultSelection) {
let deferred = promise.defer();
this.outerHTMLEditable = this._target.client.traits.editOuterHTML;
this.onNewRoot = this.onNewRoot.bind(this);
this.walker.on("new-root", this.onNewRoot);
this.nodemenu = this.panelDoc.getElementById("inspector-node-popup");
this.lastNodemenuItem = this.nodemenu.lastChild;
this._setupNodeMenu = this._setupNodeMenu.bind(this);
this._resetNodeMenu = this._resetNodeMenu.bind(this);
this.nodemenu.addEventListener("popupshowing", this._setupNodeMenu, true);
this.nodemenu.addEventListener("popuphiding", this._resetNodeMenu, true);
// Create an empty selection
this._selection = new Selection(this.walker);
this.onNewSelection = this.onNewSelection.bind(this);
this.selection.on("new-node-front", this.onNewSelection);
this.onBeforeNewSelection = this.onBeforeNewSelection.bind(this);
this.selection.on("before-new-node-front", this.onBeforeNewSelection);
this.onDetached = this.onDetached.bind(this);
this.selection.on("detached-front", this.onDetached);
this.breadcrumbs = new HTMLBreadcrumbs(this);
if (this.target.isLocalTab) {
this.browser = this.target.tab.linkedBrowser;
this.scheduleLayoutChange = this.scheduleLayoutChange.bind(this);
this.browser.addEventListener("resize", this.scheduleLayoutChange, true);
// Show a warning when the debugger is paused.
// We show the warning only when the inspector
// is selected.
this.updateDebuggerPausedWarning = function() {
let notificationBox = this._toolbox.getNotificationBox();
let notification = notificationBox.getNotificationWithValue("inspector-script-paused");
if (!notification && this._toolbox.currentToolId == "inspector" &&
this.target.isThreadPaused) {
let message = this.strings.GetStringFromName("debuggerPausedWarning.message");
notificationBox.appendNotification(message,
"inspector-script-paused", "", notificationBox.PRIORITY_WARNING_HIGH);
}
if (notification && this._toolbox.currentToolId != "inspector") {
notificationBox.removeNotification(notification);
}
if (notification && !this.target.isThreadPaused) {
notificationBox.removeNotification(notification);
}
}.bind(this);
this.target.on("thread-paused", this.updateDebuggerPausedWarning);
this.target.on("thread-resumed", this.updateDebuggerPausedWarning);
this._toolbox.on("select", this.updateDebuggerPausedWarning);
this.updateDebuggerPausedWarning();
}
this.highlighter = new Highlighter(this.target, this, this._toolbox);
let button = this.panelDoc.getElementById("inspector-inspect-toolbutton");
this.onLockStateChanged = function() {
if (this.highlighter.locked) {
button.removeAttribute("checked");
this._toolbox.raise();
} else {
button.setAttribute("checked", "true");
}
}.bind(this);
this.highlighter.on("locked", this.onLockStateChanged);
this.highlighter.on("unlocked", this.onLockStateChanged);
this._initMarkup();
this.isReady = false;
this.once("markuploaded", function() {
this.isReady = true;
// All the components are initialized. Let's select a node.
this._selection.setNodeFront(defaultSelection);
this.markup.expandNode(this.selection.nodeFront);
this.emit("ready");
deferred.resolve(this);
}.bind(this));
this.setupSearchBox();
this.setupSidebar();
return deferred.promise;
},
_onBeforeNavigate: function() {
this._defaultNode = null;
this.selection.setNodeFront(null);
this._destroyMarkup();
this.isDirty = false;
},
_getWalker: function() {
return this.inspector.getWalker().then(walker => {
this.walker = walker;
return this.inspector.getPageStyle();
}).then(pageStyle => {
this.pageStyle = pageStyle;
});
},
...
I didn't see this Promise documented anywhere in the Addon APIs, is there any documentation (or even source comments) on what this is, and how it is used?
Could it be used to add special styling or append some icons to certain elements in the DOM tree view of the Firefox DevTools Inspector?
Whenever "walker" is mentioned in the devtools code, it usually refers to the WalkerActor class in toolkit/devtools/server/actors/inspector.js.
Actors are javascript classes that are specifically made to get information from the currently inspected page and context or manipulate them.
The UI part of the tools you see as a user don't do that directly. Indeed, the devtools use a client-server protocol to communicate between the toolbox (that hosts all of the panels you use) and the actors that run as part of the page being inspected. This is what allows to use the tools to inspect remote devices.
When it comes to the WalkerActor in particular, its role is to traverse the DOM tree and give information about nodes to the inspector-panel so that it can be displayed in the tools.
What you see when you open the devtools inspector on a page is a part of the DOM tree (a part only because it's not entirely expanded and collapsed nodes haven't been retrieved yet) that has been retrieved by the WalkerActor and sent (via the protocol) to the inspector panel.
The actual UI rendering of the panel is done on the client-side (which means, the toolbox side, in comparison with the actors/page side), in browser/devtools/markupview/markup-view.js
In this file, MarkupView and MarkupContainer classes in particular are responsible for displaying nodes. They're not specifically part of the Addons API, but it should be relatively easy to get hold of them since privileged code has access to the gDevTools global variable:
Cu.import("resource://gre/modules/devtools/Loader.jsm");
let target = devtools.TargetFactory.forTab(gBrowser.selectedTab);
let toolbox = gDevTools.getToolbox(target);
let inspectorPanel = toolbox.getPanel("inspector");
inspector.markup // Returns the instance of MarkupView that is currently loaded in the inspector