Camera widget #13

hhaensel · 2022-01-13T09:42:45Z

I have turned the WebCam Demo into a camera widget.
It's not yet very fast and can certainly be optimised, perhaps by using VideoIO.jl, that's why I currently leave it in broken.

Whoever is interested, please try out and comment here.

The text was updated successfully, but these errors were encountered:

hhaensel · 2022-01-13T09:45:00Z

You have to add Electron.jl to your packages in order to use this app.

Maybe you have to adapt the camera name according to your system, it's not yet very comfortable.

The widget can be moved by dragging the picture.

AbhimanyuAryan · 2022-01-13T14:07:07Z

@hhaensel looks like in my case can't access camera. I'm on Macbook. Do I need to allow some sort of camera permission? What OS are you on?

 @ Genie.AppServer ~/.julia/packages/Genie/Qtv4L/src/AppServer.jl:274
┌ Error: UndefRefError: access to undefined reference
│ Stacktrace:
│   [1] getproperty
│     @ ./Base.jl:33 [inlined]
│   [2] getindex
│     @ ./refvalue.jl:56 [inlined]
│   [3] (::var"#16#17")()
│     @ Main ~/Desktop/WebCam.jl:170
│   [4] match_routes(req::HTTP.Messages.Request, res::HTTP.Messages.Response, params::Genie.Router.Params{Any})
│     @ Genie.Router ~/.julia/packages/Genie/Qtv4L/src/Router.jl:470
│   [5] route_request(req::HTTP.Messages.Request, res::HTTP.Messages.Response)
│     @ Genie.Router ~/.julia/packages/Genie/Qtv4L/src/Router.jl:149
│   [6] handle_request(req::HTTP.Messages.Request, res::HTTP.Messages.Response)
│     @ Genie.AppServer ~/.julia/packages/Genie/Qtv4L/src/AppServer.jl:235
│   [7] #24
│     @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/macros.jl:127 [inlined]
│   [8] run_work_thunk(thunk::Genie.AppServer.var"#24#25"{HTTP.Messages.Request, HTTP.Messages.Response}, print_error::Bool)
│     @ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:63
│   [9] remotecall_fetch(::Function, ::Distributed.LocalProcess; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
│     @ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:379
│  [10] remotecall_fetch(::Function, ::Distributed.LocalProcess)
│     @ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:379
│  [11] remotecall_fetch(::Function, ::Int64; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
│     @ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:421
│  [12] remotecall_fetch(::Function, ::Int64)
│     @ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:421
│  [13] setup_http_listener(req::HTTP.Messages.Request, res::HTTP.Messages.Response)
│     @ Genie.AppServer ~/.julia/packages/Genie/Qtv4L/src/AppServer.jl:262
│  [14] setup_http_listener
│     @ ~/.julia/packages/Genie/Qtv4L/src/AppServer.jl:261 [inlined]
│  [15] handle
│     @ ~/.julia/packages/HTTP/aTjcj/src/Handlers.jl:254 [inlined]
│  [16] handle(::HTTP.Handlers.RequestHandlerFunction{typeof(Genie.AppServer.setup_http_listener)}, ::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}})
│     @ HTTP.Handlers ~/.julia/packages/HTTP/aTjcj/src/Handlers.jl:277
│  [17] setup_http_streamer(http::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}})
│     @ Genie.AppServer ~/.julia/packages/Genie/Qtv4L/src/AppServer.jl:251
│  [18] (::Genie.AppServer.var"#7#14"{Int64})(http::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}})
│     @ Genie.AppServer ~/.julia/packages/Genie/Qtv4L/src/AppServer.jl:117
│  [19] macro expansion
│     @ ~/.julia/packages/HTTP/aTjcj/src/Servers.jl:415 [inlined]
│  [20] (::HTTP.Servers.var"#13#14"{Genie.AppServer.var"#7#14"{Int64}, HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}, HTTP.Servers.Server{Nothing, Sockets.TCPServer}, HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}}})()
│     @ HTTP.Servers ./task.jl:411
│
└ @ Genie.AppServer ~/.julia/packages/Genie/Qtv4L/src/AppServer.jl:274
 ~/Desktop

hhaensel · 2022-01-13T20:19:57Z

I'm on windows 10. You can try to use VideoIO.jl to find out the name of your camera device:

using VideoIO
VideoIO.CAMERA_DEVICES

Then put the name of your camera in place of /dev/video0.

Good luck!

hhaensel · 2022-01-13T20:20:49Z

P.S.: I'm struggling to get VideoIO running on my windows PC.

AbhimanyuAryan · 2022-01-13T20:22:58Z

I'm on windows 10. You can try to use VideoIO.jl to find out the name of your camera device:
using VideoIO
VideoIO.CAMERA_DEVICES
Then put the name of your camera in place of /dev/video0.

Good luck!

Will look into it. Thanks 🙃

hhaensel · 2022-01-13T20:41:37Z

Just found out how to get VideoIO working:

import Base.unsafe_convert
unsafe_convert(::Type{Ptr{Ptr{T}}}, ad::VideoIO.AVDict) where T = unsafe_convert(Ptr{Ptr{T}}, getfield(ad, :ref_ptr_dict))

So there is some probability that we will switch to VideoIO soon.

hhaensel · 2022-01-14T11:45:48Z

Will look into it. Thanks 🙃

You could try

const FMT = "avfoundation"

or

const FMT = "qtkit"

The next version will automate this process ...

hhaensel · 2022-01-14T18:39:59Z

Just submitted a correction for webcam that should make it functional on all platforms.
Another version based on VideoIO is released as webcam2.jl. It's slower, as it does the conversion to png in Julia whereas the original version uses ffmpeg conversion.

@essenciary , @AbhimanyuAryan
Please test. If both are functional on your machines, we might move it to the working demos.

AbhimanyuAryan · 2022-01-15T06:11:52Z

@hhaensel webcam2.jl works. Asked for facetime permission but it's very slow. Infact it only took first frame and didn't update new frames. Toggle off turns off the camera.

hhaensel · 2022-01-15T07:43:44Z

You have to decrease the FPS_CLIENT in this case.
Webcam.jl should also work for you now. Please also decrease FPS_CLIENTthere. Yesterday I tried it it on a slower machine and it only worked with FPS_CLIENT=10

hhaensel · 2022-01-15T07:45:14Z

Toggle turns off the camera so that other programs it like zoom or teams can use it

AbhimanyuAryan · 2022-01-17T04:50:03Z

will check :)

hhaensel · 2022-01-17T06:52:21Z

Please check WebCam3.jl
It now supports two updatemodes: webchannels and url, implements cameras in a cleaner way and supports multiple cameras.
WebChannel updatemode doesn't populate the log and is more secure.

AbhimanyuAryan · 2022-01-18T16:08:47Z

yes @hhaensel checking ...

hhaensel · 2022-01-21T12:19:39Z

Has anyone tested? Can we move it to Advanced?

AbhimanyuAryan · 2022-01-21T12:48:48Z

I haven't had time. I'll test tomorrow :)

hhaensel · 2022-01-21T13:42:25Z

The latest feature of WebCam3.jl is autosizing of the widget if you drag the side handles. The edge handles don't work properly, but that's not a big issue as they are anyhow difficult to find.
For debugging or easier finding of the window handles I have introduced a backgound color on mouse hover. It's set to transparent by default.

AbhimanyuAryan · 2022-01-22T10:41:42Z

I'm running on macbook m1(16gb ram)

change FPS client but image is still stuck in electron widget for web2.jl
web.jl: [AVFoundation indev @ 0x7f884b705980] Video device not found
video=FaceTime HD Camera: Input/output error crashes
webcam3.jl

 paused, 0 bytes waiting), Process(`/Users/abhi/.julia/artifacts/3ba739a4ed073587d96415faecef10a5ccea63ba/Julia.app/Contents/MacOS/Julia /Users/abhi/.julia/packages/Electron/ILvWR/src/main.js /var/folders/0j/d0xd373d2tb7zdzcsqrk3mpw0000gn/T/jlel-a25622e87b6f11ec224cb3815cd79e21 /var/folders/0j/d0xd373d2tb7zdzcsqrk3mpw0000gn/T/jlel-sn-a2562f367b6f11ec0e38d7a7073e8830 s049w1CM47lShNmbrUMDFtoDMbobaHcsdiDdoiY599qzUJ5qRaOFMDaDwvt0YQAZlKFu26h89FRtTlshIsxIYtVnKTNkgJOgNAZpplVILStLgyZK/BHDZPRce2M5dLE2YWPlQ/2bbqnfiV+oWosMVWXrxQdHO8XdJ3WwTrFejA4=`, ProcessRunning), [1 window]), 1, true, Channel{Any}(128))

julia> Received > 3 system signals, hard exiting
[ Info: starting camera with 'video=FaceTime HD Camera'
[AVFoundation indev @ 0x7f855af05980] Video device not found
video=FaceTime HD Camera: Input/output error

hhaensel · 2022-01-23T00:29:53Z

So we have to solve two problems:

get VideoIO running on your m1. Did you try running the VideoIO demo withh makie which is on the VideoIO.jl page?
find the root cause of the crash of ffmpeg (webcam3.jl). Maybe it's related to wrong values of sx, sy or fps? If you can use VideoIO, you might be able to find out these values by examining the picture it grabbed. Do you have a new m1 with the 1080p facetime camera or do you have 720p?

AbhimanyuAryan · 2022-01-23T02:47:20Z

Some weird issue with VideoIO examples:

julia> include("videomakie.jl")
ERROR: LoadError: MethodError: no method matching unsafe_convert(::Type{Ptr{Ptr{VideoIO.libffmpeg.AVDictionary}}}, ::VideoIO.AVDict)
Closest candidates are:
  unsafe_convert(::Type{Ptr{T}}, ::StaticArrays.MArray{S, T, N, L} where {N, L}) where {S, T} at /Users/abhi/.julia/packages/StaticArrays/A0XaR/src/MArray.jl:109
  unsafe_convert(::Type{Ptr{T}}, ::FFTW.FakeArray{T, N} where N) where T at /Users/abhi/.julia/packages/FFTW/pHa9y/src/fft.jl:121
  unsafe_convert(::Type{Ptr{T}}, ::Ptr{Tuple{Vararg{T, N}}}) where {N, T} at refpointer.jl:176
  ...
Stacktrace:
 [1] avformat_open_input(ps::VideoIO.NestedCStruct{VideoIO.libffmpeg.AVFormatContext}, url::String, fmt::Ptr{VideoIO.libffmpeg.AVInputFormat}, options::VideoIO.AVDict)
   @ VideoIO.libffmpeg ~/.julia/packages/VideoIO/EsW3Z/lib/libffmpeg.jl:8571
 [2] open_avinput(avin::VideoIO.AVInput{String}, source::String, input_format::Ptr{VideoIO.libffmpeg.AVInputFormat}, options::VideoIO.AVDict)
   @ VideoIO ~/.julia/packages/VideoIO/EsW3Z/src/avio.jl:195
 [3] VideoIO.AVInput(source::String, input_format::Ptr{VideoIO.libffmpeg.AVInputFormat}, options::VideoIO.AVDict; avio_ctx_buffer_size::Int64)
   @ VideoIO ~/.julia/packages/VideoIO/EsW3Z/src/avio.jl:218
 [4] VideoIO.AVInput(source::String, input_format::Ptr{VideoIO.libffmpeg.AVInputFormat}, options::VideoIO.AVDict)
   @ VideoIO ~/.julia/packages/VideoIO/EsW3Z/src/avio.jl:207
 [5] opencamera(::String, ::Ptr{VideoIO.libffmpeg.AVInputFormat}, ::VideoIO.AVDict; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ VideoIO ~/.julia/packages/VideoIO/EsW3Z/src/avio.jl:1016
 [6] opencamera(::String, ::Ptr{VideoIO.libffmpeg.AVInputFormat}, ::VideoIO.AVDict) (repeats 2 times)
   @ VideoIO ~/.julia/packages/VideoIO/EsW3Z/src/avio.jl:1016
 [7] top-level scope
   @ ~/Git/VideoIO/videomakie.jl:4
 [8] include(fname::String)
   @ Base.MainInclude ./client.jl:444
 [9] top-level scope
   @ REPL[9]:1
in expression starting at /Users/abhi/Git/VideoIO/videomakie.jl:4

followed this for solution. No luck: https://discourse.julialang.org/t/running-external-programs-in-windows/72353/6?u=abhimanyuaryan

Need to dig deeper

I have 720p m1 air

hhaensel · 2022-01-23T07:49:22Z

Try to execute the two lines that are also in the webcam2 and webcam3. Before you call open camera.

import VideoIO

import Base.cconvert
Base.cconvert(::Type{Ptr{Ptr{VideoIO.AVDictionary}}}, d::VideoIO.AVDict) = d.ref_ptr_dict

cam = opencamera()

AbhimanyuAryan · 2022-01-27T16:06:24Z

will check/debug on weekend :)

AbhimanyuAryan · 2022-01-31T08:30:02Z

Not very clear what's causing this: JuliaIO/VideoIO.jl#343 says it's the frame rate. Have you tried running this default example?

import VideoIO, GLMakie
import Base.cconvert
Base.cconvert(::Type{Ptr{Ptr{VideoIO.AVDictionary}}}, d::VideoIO.AVDict) = d.ref_ptr_dict

cam = VideoIO.opencamera()

img = read(cam)
scene = GLMakie.Scene(resolution = size(img'))
makieimg = GLMakie.image!(scene, img)
GLMakie.rotate!(scene, -0.5pi)
display(scene)

while isopen(scene)
    read!(cam, img)
    makieimg.image = img
    sleep(1/VideoIO.framerate(cam))
end

close(cam)

ERROR: LoadError: Could not scale frame
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] exec!
   @ ~/.julia/dev/VideoIO/src/frame_graph.jl:109 [inlined]
 [3] execute_graph!
   @ ~/.julia/dev/VideoIO/src/avio.jl:456 [inlined]
 [4] _retrieve!(r::VideoIO.VideoReader{true, VideoIO.SwsTransform, String}, buf::PermutedDimsArray{ColorTypes.RGB{FixedPointNumbers.N0f8}, 2, (2, 1), (2, 1), Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}})
   @ VideoIO ~/.julia/dev/VideoIO/src/avio.jl:467
 [5] retrieve!
   @ ~/.julia/dev/VideoIO/src/avio.jl:478 [inlined]
 [6] read!(r::VideoIO.VideoReader{true, VideoIO.SwsTransform, String}, buf::PermutedDimsArray{ColorTypes.RGB{FixedPointNumbers.N0f8}, 2, (2, 1), (2, 1), Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}})
   @ VideoIO ~/.julia/dev/VideoIO/src/avio.jl:637
 [7] top-level scope
   @ ~/Git/VideoIO/videomakie.jl:14
 [8] include(fname::String)
   @ Base.MainInclude ./client.jl:444
 [9] top-level scope
   @ REPL[1]:1
in expression starting at /Users/abhi/Git/VideoIO/videomakie.jl:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Camera widget #13

Camera widget #13

hhaensel commented Jan 13, 2022

hhaensel commented Jan 13, 2022 •

edited

Loading

AbhimanyuAryan commented Jan 13, 2022 •

edited

Loading

hhaensel commented Jan 13, 2022

hhaensel commented Jan 13, 2022

AbhimanyuAryan commented Jan 13, 2022 •

edited

Loading

hhaensel commented Jan 13, 2022

hhaensel commented Jan 14, 2022

hhaensel commented Jan 14, 2022

AbhimanyuAryan commented Jan 15, 2022

hhaensel commented Jan 15, 2022

hhaensel commented Jan 15, 2022

AbhimanyuAryan commented Jan 17, 2022

hhaensel commented Jan 17, 2022 •

edited

Loading

AbhimanyuAryan commented Jan 18, 2022

hhaensel commented Jan 21, 2022

AbhimanyuAryan commented Jan 21, 2022

hhaensel commented Jan 21, 2022

AbhimanyuAryan commented Jan 22, 2022 •

edited

Loading

hhaensel commented Jan 23, 2022

AbhimanyuAryan commented Jan 23, 2022 •

edited

Loading

hhaensel commented Jan 23, 2022 •

edited

Loading

AbhimanyuAryan commented Jan 27, 2022

AbhimanyuAryan commented Jan 31, 2022

Camera widget #13

Camera widget #13

Comments

hhaensel commented Jan 13, 2022

hhaensel commented Jan 13, 2022 • edited Loading

AbhimanyuAryan commented Jan 13, 2022 • edited Loading

hhaensel commented Jan 13, 2022

hhaensel commented Jan 13, 2022

AbhimanyuAryan commented Jan 13, 2022 • edited Loading

hhaensel commented Jan 13, 2022

hhaensel commented Jan 14, 2022

hhaensel commented Jan 14, 2022

AbhimanyuAryan commented Jan 15, 2022

hhaensel commented Jan 15, 2022

hhaensel commented Jan 15, 2022

AbhimanyuAryan commented Jan 17, 2022

hhaensel commented Jan 17, 2022 • edited Loading

AbhimanyuAryan commented Jan 18, 2022

hhaensel commented Jan 21, 2022

AbhimanyuAryan commented Jan 21, 2022

hhaensel commented Jan 21, 2022

AbhimanyuAryan commented Jan 22, 2022 • edited Loading

hhaensel commented Jan 23, 2022

AbhimanyuAryan commented Jan 23, 2022 • edited Loading

hhaensel commented Jan 23, 2022 • edited Loading

AbhimanyuAryan commented Jan 27, 2022

AbhimanyuAryan commented Jan 31, 2022

hhaensel commented Jan 13, 2022 •

edited

Loading

AbhimanyuAryan commented Jan 13, 2022 •

edited

Loading

AbhimanyuAryan commented Jan 13, 2022 •

edited

Loading

hhaensel commented Jan 17, 2022 •

edited

Loading

AbhimanyuAryan commented Jan 22, 2022 •

edited

Loading

AbhimanyuAryan commented Jan 23, 2022 •

edited

Loading

hhaensel commented Jan 23, 2022 •

edited

Loading