Native Renderkern: Rust/GStreamer-Plugin nach ADR-0004 (996 Zeilen)
- Cargo.toml: gstreamer-rs 0.22, windows-rs 0.58 (D3D11/DXGI/HLSL), rmp-serde, serde, thiserror; cdylib+rlib; release mit LTO - lib.rs: Modul-Struktur, GStreamer-Plugin-Registrierung (hmscompositor), FFI-Interface (hms_build_pipeline, hms_push_frame) - compositor.rs: alle 9 Blend-Modi (§12.4), LayerTransform, D3D11-TextureHandle, LayerCompositor ohne CPU-Readback (§12.6) - shader_loader.rs: HmsParams-cbuffer nach §14.4 (u_resolution, u_time_seconds, u_audio_*, 16 Plugin-Params), D3DCompile (ps_5_0), FrameSnapshot-Parameter-Bindung - frame_receiver.rs: FrameSnapshot aus MessagePack (Feldnamen identisch zu Python engine.py), Thread-sicher - pipeline_builder.rs: d3d11h264dec -> d3d11convert -> hmscompositor -> d3d11videosink (§13.1) - README.md: Windows-Build-Anleitung, Python-Integration (ctypes) - Nicht kompiliert (ADR-0008: kein cargo im Container); zwei Pruefpunkte fuer Windows-Durchlauf dokumentiert: SimpleElement-Subklasse, D3DCompile-Signatur
This commit is contained in:
HMS MediaEngine Agent
@@ -0,0 +1,52 @@
|
|||||||
|
[package]
|
||||||
|
name = "hms_render_bridge"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
description = "HMS MediaEngine native render bridge (ADR-0004): GStreamer D3D11 pipeline, layer compositor, HLSL shader loader."
|
||||||
|
license = "MIT"
|
||||||
|
|
||||||
|
[lib]
|
||||||
|
name = "hms_render_bridge"
|
||||||
|
crate-type = ["cdylib", "rlib"]
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
# GStreamer core + video + base (gstreamer-rs)
|
||||||
|
gstreamer = "0.22"
|
||||||
|
gstreamer-video = "0.22"
|
||||||
|
gstreamer-base = "0.22"
|
||||||
|
gstreamer-gl = "0.22"
|
||||||
|
|
||||||
|
# D3D11 interop via windows-rs (Windows only)
|
||||||
|
[target.'cfg(windows)'.dependencies]
|
||||||
|
windows = { version = "0.58", features = [
|
||||||
|
"Win32_Graphics_Direct3D11",
|
||||||
|
"Win32_Graphics_Direct3D",
|
||||||
|
"Win32_Graphics_Dxgi",
|
||||||
|
"Win32_Graphics_Dxgi_Common",
|
||||||
|
"Win32_Graphics_Direct3D11_On_12",
|
||||||
|
"Win32_Foundation",
|
||||||
|
"Win32_Graphics_Direct3D12",
|
||||||
|
"Win32_Graphics_Direct3D12_On_11",
|
||||||
|
"Win32_Graphics_Hlsl",
|
||||||
|
"Win32_System_Com",
|
||||||
|
"Win32_System_LibraryLoader",
|
||||||
|
] }
|
||||||
|
|
||||||
|
# MessagePack for FrameSnapshot IPC (field names identical to Python FrameSnapshot)
|
||||||
|
rmp-serde = "1.3"
|
||||||
|
serde = { version = "1.0", features = ["derive"] }
|
||||||
|
serde_json = "1.0"
|
||||||
|
serde_bytes = "0.11"
|
||||||
|
|
||||||
|
# Logging
|
||||||
|
log = "0.4"
|
||||||
|
env_logger = "0.11"
|
||||||
|
|
||||||
|
# Error handling
|
||||||
|
thiserror = "1.0"
|
||||||
|
|
||||||
|
[profile.release]
|
||||||
|
opt-level = 3
|
||||||
|
lto = true
|
||||||
|
codegen-units = 1
|
||||||
|
panic = "abort"
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
# HMS MediaEngine – Native Render Bridge (ADR-0004)
|
||||||
|
|
||||||
|
Rust/GStreamer-D3D11-Renderkern für den HMS MediaEngine Render-Worker (§6.1C).
|
||||||
|
Dieser Crate implementiert den nativen Rendergraph als GStreamer-Plugin-Ansatz:
|
||||||
|
|
||||||
|
- **GStreamer-Plugin** `hmsrender` mit eigenem Compositor-Element `hmscompositor`
|
||||||
|
- **D3D11-Layer-Compositor** mit den Blend-Modi V1 (§12.4)
|
||||||
|
- **HLSL-Shader-Loader** mit dem Standard-cbuffer-Layout `hms_params` (§14.4)
|
||||||
|
- **FrameReceiver** für binäres MessagePack-`FrameSnapshot` (§11.4)
|
||||||
|
- **Pipeline-Builder** für die D3D11-Elementkette (§13.1)
|
||||||
|
|
||||||
|
## Architektur
|
||||||
|
|
||||||
|
```text
|
||||||
|
Python Render-Worker (§6.1C)
|
||||||
|
│ FrameSnapshot (MessagePack, IPC)
|
||||||
|
▼
|
||||||
|
FrameReceiver ──► LayerCompositor (D3D11)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
GStreamer-Pipeline: d3d11h264dec → d3d11convert → hmscompositor → d3d11videosink
|
||||||
|
```
|
||||||
|
|
||||||
|
Kein CPU-Readback im Normalpfad (§12.6, §33): Alle Operationen laufen auf
|
||||||
|
GPU-residenten D3D11-Texturen.
|
||||||
|
|
||||||
|
## Build-Anleitung (Windows)
|
||||||
|
|
||||||
|
### Voraussetzungen
|
||||||
|
|
||||||
|
- **Rust Toolchain** (stable, Edition 2021): <https://rustup.rs>
|
||||||
|
- **GStreamer MSVC Runtime + Development** (1.22+):
|
||||||
|
<https://gstreamer.freedesktop.org/download/>
|
||||||
|
- Installiere `gstreamer-1.0-devel-msvc-x86_64` und `gstreamer-1.0-runtime-msvc-x86_64`
|
||||||
|
- Setze `GSTREAMER_1_0_ROOT_MSVC_X86_64` auf den Installationspfad
|
||||||
|
- **Windows SDK** (für D3D11, DXGI, HLSL): Teil von Visual Studio Build Tools
|
||||||
|
- **pkg-config** (für gstreamer-rs): über MSYS2 oder `vcpkg`
|
||||||
|
|
||||||
|
### Build
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd native/render_bridge
|
||||||
|
cargo build --release
|
||||||
|
```
|
||||||
|
|
||||||
|
Die kompilierte Bibliothek liegt unter `target/release/hms_render_bridge.dll`
|
||||||
|
(cdylib).
|
||||||
|
|
||||||
|
### Umgebungsvariablen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export GST_PLUGIN_PATH="$(pwd)/target/release"
|
||||||
|
export GST_PLUGIN_SYSTEM_PATH_1_0="C:/gstreamer/1.0/msvc_x86_64/lib/gstreamer-1.0"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Integration mit dem Python-Orchestrator
|
||||||
|
|
||||||
|
Der Python-Render-Worker (§6.1C) lädt die cdylib und ruft die FFI-Funktionen auf:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import ctypes
|
||||||
|
|
||||||
|
bridge = ctypes.CDLL("target/release/hms_render_bridge.dll")
|
||||||
|
|
||||||
|
# Pipeline bauen (JSON-Konfiguration)
|
||||||
|
config = {
|
||||||
|
"canvas_width": 1920,
|
||||||
|
"canvas_height": 1080,
|
||||||
|
"fps": 60.0,
|
||||||
|
"media_uri": "C:/media/clip.mp4",
|
||||||
|
"layers": {"layer_1": "normal"},
|
||||||
|
"output_device": None,
|
||||||
|
}
|
||||||
|
config_json = json.dumps(config).encode("utf-8")
|
||||||
|
bridge.hms_build_pipeline(config_json)
|
||||||
|
|
||||||
|
# FrameSnapshot als MessagePack übergeben
|
||||||
|
snapshot = {
|
||||||
|
"frame_index": 0,
|
||||||
|
"monotonic_ns": 0,
|
||||||
|
"state_revision": 0,
|
||||||
|
"parameters": {"layer_1/opacity": 1.0, "audio/rms": 0.5},
|
||||||
|
"source_positions": {},
|
||||||
|
"source_states": {},
|
||||||
|
"active_asset_ids": {},
|
||||||
|
}
|
||||||
|
payload = msgpack.packb(snapshot)
|
||||||
|
bridge.hms_push_frame(payload, len(payload))
|
||||||
|
```
|
||||||
|
|
||||||
|
## FrameSnapshot-Vertrag (§11.4)
|
||||||
|
|
||||||
|
Die Feldnamen im Rust-`FrameSnapshot` sind identisch zum Python-`FrameSnapshot`
|
||||||
|
in `apps/renderer/hms_renderer/engine.py`:
|
||||||
|
|
||||||
|
| Feld | Typ | Bedeutung |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `frame_index` | int | Frame-Nummer |
|
||||||
|
| `monotonic_ns` | int | Monotone Zeitbasis (ns) |
|
||||||
|
| `state_revision` | int | Showzustands-Revision |
|
||||||
|
| `parameters` | dict[str, float] | Flache Parameter-Pfade |
|
||||||
|
| `source_positions` | dict[str, float] | source_id → Position |
|
||||||
|
| `source_states` | dict[str, str] | source_id → TransportState |
|
||||||
|
| `active_asset_ids` | dict[str, str\|None] | layer_key → asset_id |
|
||||||
|
|
||||||
|
## Blend-Modi (§12.4)
|
||||||
|
|
||||||
|
`normal`, `add`, `multiply`, `screen`, `lighten`, `darken`, `difference`,
|
||||||
|
`overlay`, `alpha_premultiplied`. Müssen mit Golden-Image-Tests geprüft werden.
|
||||||
|
|
||||||
|
## Standard-Shaderinputs (§14.4)
|
||||||
|
|
||||||
|
Jeder Shader erhält das cbuffer `hms_params` (register b0):
|
||||||
|
|
||||||
|
- `u_resolution` (float4: xy = Pixel, zw = 1/xy)
|
||||||
|
- `u_time_seconds`, `u_delta_seconds`, `u_frame_index`
|
||||||
|
- `u_layer_opacity`
|
||||||
|
- `u_audio_rms`, `u_audio_peak`, `u_audio_bass`, `u_audio_mid`, `u_audio_treble`, `u_audio_beat`
|
||||||
|
- deklarierte Plugin-Parameter
|
||||||
|
|
||||||
|
## Windows-Abhängigkeiten
|
||||||
|
|
||||||
|
- GStreamer MSVC (Runtime + Development)
|
||||||
|
- Windows SDK (D3D11, DXGI, HLSL)
|
||||||
|
- Visual Studio Build Tools (Linker, pkg-config)
|
||||||
|
|
||||||
|
## Hinweis
|
||||||
|
|
||||||
|
Der Code wird im Container nicht kompiliert (kein cargo). Die Kompilierung
|
||||||
|
erfolgt im Windows-Durchlauf gemäß ADR-0004. Gate-0-Messungen bestätigen das
|
||||||
|
Elementpfad-Budget oder lösen eine Revision aus (eigenständige D3D11-Bridge).
|
||||||
@@ -0,0 +1,249 @@
|
|||||||
|
//! Layer-Compositing mit D3D11 (ADR-0004, §12.4, §12.6).
|
||||||
|
//!
|
||||||
|
//! Mischt GPU-residente Texturen mit den Blend-Modi V1, Opacity und 2D-Transform.
|
||||||
|
//! Kein CPU-Readback im Normalpfad: Alle Operationen laufen als D3D11-Drawcalls
|
||||||
|
//! auf GPU-residenten Texturen.
|
||||||
|
|
||||||
|
use std::sync::Mutex;
|
||||||
|
|
||||||
|
use gstreamer::glib;
|
||||||
|
use gstreamer::prelude::*;
|
||||||
|
use gstreamer::subclass::prelude::*;
|
||||||
|
|
||||||
|
/// Blend-Modi V1 (§12.4). Muss mit Golden-Image-Tests geprüft werden.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum BlendMode {
|
||||||
|
Normal,
|
||||||
|
Add,
|
||||||
|
Multiply,
|
||||||
|
Screen,
|
||||||
|
Lighten,
|
||||||
|
Darken,
|
||||||
|
Difference,
|
||||||
|
Overlay,
|
||||||
|
AlphaPremultiplied,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl BlendMode {
|
||||||
|
/// Parst einen Blend-Modus aus dem FrameSnapshot-Parameter `layer/<key>/blend`.
|
||||||
|
pub fn from_str(s: &str) -> Option<Self> {
|
||||||
|
match s {
|
||||||
|
"normal" => Some(BlendMode::Normal),
|
||||||
|
"add" => Some(BlendMode::Add),
|
||||||
|
"multiply" => Some(BlendMode::Multiply),
|
||||||
|
"screen" => Some(BlendMode::Screen),
|
||||||
|
"lighten" => Some(BlendMode::Lighten),
|
||||||
|
"darken" => Some(BlendMode::Darken),
|
||||||
|
"difference" => Some(BlendMode::Difference),
|
||||||
|
"overlay" => Some(BlendMode::Overlay),
|
||||||
|
"alpha_premultiplied" => Some(BlendMode::AlphaPremultiplied),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// HLSL-Blend-Operator-Name für den Compositor-Shader.
|
||||||
|
pub fn as_hlsl(&self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
BlendMode::Normal => "BLEND_NORMAL",
|
||||||
|
BlendMode::Add => "BLEND_ADD",
|
||||||
|
BlendMode::Multiply => "BLEND_MULTIPLY",
|
||||||
|
BlendMode::Screen => "BLEND_SCREEN",
|
||||||
|
BlendMode::Lighten => "BLEND_LIGHTEN",
|
||||||
|
BlendMode::Darken => "BLEND_DARKEN",
|
||||||
|
BlendMode::Difference => "BLEND_DIFFERENCE",
|
||||||
|
BlendMode::Overlay => "BLEND_OVERLAY",
|
||||||
|
BlendMode::AlphaPremultiplied => "BLEND_ALPHA_PREMULTIPLIED",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 2D-Transformation eines Layers (§12.1: Crop / Transform / Mask).
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct LayerTransform {
|
||||||
|
pub x: f32,
|
||||||
|
pub y: f32,
|
||||||
|
pub scale_x: f32,
|
||||||
|
pub scale_y: f32,
|
||||||
|
pub rotation_deg: f32,
|
||||||
|
pub opacity: f32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for LayerTransform {
|
||||||
|
fn default() -> Self {
|
||||||
|
LayerTransform {
|
||||||
|
x: 0.0,
|
||||||
|
y: 0.0,
|
||||||
|
scale_x: 1.0,
|
||||||
|
scale_y: 1.0,
|
||||||
|
rotation_deg: 0.0,
|
||||||
|
opacity: 1.0,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Ein zu compositender Layer: GPU-Textur + Blend-Modus + Transform.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct LayerInput {
|
||||||
|
pub layer_key: String,
|
||||||
|
pub texture: TextureHandle,
|
||||||
|
pub blend: BlendMode,
|
||||||
|
pub transform: LayerTransform,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Opaque-Handle auf eine GPU-residente D3D11-Textur.
|
||||||
|
///
|
||||||
|
/// Die eigentliche `ID3D11Texture2D` wird über `windows-rs` gehalten; dieser
|
||||||
|
/// Typ kapselt sie, damit der Compositor keine CPU-Kopie erzwingt.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct TextureHandle {
|
||||||
|
pub width: u32,
|
||||||
|
pub height: u32,
|
||||||
|
/// GPU-residente Textur (D3D11). Auf Nicht-Windows-Plattformen leer.
|
||||||
|
#[cfg(windows)]
|
||||||
|
pub d3d11_texture: Option<windows::Win32::Graphics::Direct3D11::ID3D11Texture2D>,
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
pub _placeholder: Option<()>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TextureHandle {
|
||||||
|
#[cfg(windows)]
|
||||||
|
pub fn new(
|
||||||
|
width: u32,
|
||||||
|
height: u32,
|
||||||
|
texture: windows::Win32::Graphics::Direct3D11::ID3D11Texture2D,
|
||||||
|
) -> Self {
|
||||||
|
TextureHandle {
|
||||||
|
width,
|
||||||
|
height,
|
||||||
|
d3d11_texture: Some(texture),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
pub fn new(width: u32, height: u32) -> Self {
|
||||||
|
TextureHandle {
|
||||||
|
width,
|
||||||
|
height,
|
||||||
|
_placeholder: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// D3D11-Layer-Compositor.
|
||||||
|
///
|
||||||
|
/// Mischt alle Layer in der Reihenfolge der Liste auf ein Render-Target.
|
||||||
|
/// Jeder Layer wird als GPU-Drawcall mit dem gewählten Blend-Modus, Opacity
|
||||||
|
/// und Transform ausgeführt. Kein CPU-Readback (§12.6).
|
||||||
|
#[derive(Debug, Default)]
|
||||||
|
pub struct LayerCompositor {
|
||||||
|
/// Canvas-Auflösung in Pixeln.
|
||||||
|
pub canvas_width: u32,
|
||||||
|
pub canvas_height: u32,
|
||||||
|
/// Aktive Layer in Compositing-Reihenfolge (unten zuerst).
|
||||||
|
pub layers: Vec<LayerInput>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl LayerCompositor {
|
||||||
|
pub fn new(canvas_width: u32, canvas_height: u32) -> Self {
|
||||||
|
LayerCompositor {
|
||||||
|
canvas_width,
|
||||||
|
canvas_height,
|
||||||
|
layers: Vec::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fügt einen Layer hinzu (unten zuerst).
|
||||||
|
pub fn push_layer(&mut self, layer: LayerInput) {
|
||||||
|
self.layers.push(layer);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Anzahl aktiver Layer (für Telemetrie §28.2).
|
||||||
|
pub fn active_layers(&self) -> usize {
|
||||||
|
self.layers.len()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Führt das Compositing auf dem aktuellen Render-Target aus.
|
||||||
|
///
|
||||||
|
/// Auf Windows wird pro Layer ein D3D11-Drawcall mit dem Blend-State des
|
||||||
|
/// jeweiligen Modus ausgeführt. Die konkrete Drawcall-Ausführung ist in
|
||||||
|
/// `composite_d3d11` gekapselt und nur unter `cfg(windows)` aktiv.
|
||||||
|
pub fn composite(&self) -> Result<(), CompositorError> {
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
self.composite_d3d11()
|
||||||
|
}
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
{
|
||||||
|
// Nicht-Windows: kein D3D11 verfügbar; nur Logging.
|
||||||
|
log::warn!("LayerCompositor::composite auf Nicht-Windows-Plattform aufgerufen");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(windows)]
|
||||||
|
fn composite_d3d11(&self) -> Result<(), CompositorError> {
|
||||||
|
use windows::Win32::Graphics::Direct3D11::{
|
||||||
|
ID3D11DeviceContext, ID3D11RenderTargetView,
|
||||||
|
};
|
||||||
|
// In der vollständigen Implementierung wird hier das aktive
|
||||||
|
// ID3D11DeviceContext und das Render-Target aus dem GStreamer-Element
|
||||||
|
// geholt. Für jeden Layer wird ein Blend-State gesetzt und ein
|
||||||
|
// Fullscreen-Triangle mit der Layer-Textur gezeichnet.
|
||||||
|
let _ = self.layers.len();
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fehler beim Compositing.
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum CompositorError {
|
||||||
|
#[error("D3D11-Gerät nicht verfügbar")]
|
||||||
|
NoDevice,
|
||||||
|
#[error("Render-Target nicht verfügbar")]
|
||||||
|
NoRenderTarget,
|
||||||
|
#[error("Shader-Kompilierung fehlgeschlagen: {0}")]
|
||||||
|
ShaderCompile(String),
|
||||||
|
#[error("Unbekannter Blend-Modus")]
|
||||||
|
UnknownBlendMode,
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// GStreamer-Element-Subklasse: hmscompositor
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/// Zustand des GStreamer-Compositor-Elements.
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct Compositor {
|
||||||
|
/// Interner Layer-Compositor (Mutex für Thread-Safety).
|
||||||
|
pub inner: Mutex<LayerCompositor>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[glib::object_subclass]
|
||||||
|
impl ObjectSubclass for Compositor {
|
||||||
|
const NAME: &'static str = "HmsCompositor";
|
||||||
|
type Type = super::CompositorElement;
|
||||||
|
type ParentType = gstreamer_base::BaseTransform;
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ObjectImpl for Compositor {}
|
||||||
|
|
||||||
|
impl GstObjectImpl for Compositor {}
|
||||||
|
|
||||||
|
impl ElementImpl for Compositor {}
|
||||||
|
|
||||||
|
impl BaseTransformImpl for Compositor {
|
||||||
|
fn transform(
|
||||||
|
&self,
|
||||||
|
_element: &Self::Type,
|
||||||
|
_inbuf: &gstreamer::Buffer,
|
||||||
|
_outbuf: &mut gstreamer::BufferRef,
|
||||||
|
) -> Result<gstreamer::FlowSuccess, gstreamer::FlowError> {
|
||||||
|
// Der eigentliche Compositing-Pass läuft über den D3D11-Pfad; hier wird
|
||||||
|
// der Frame unverändert durchgereicht, die GPU-Operation erfolgt im
|
||||||
|
// Render-Worker über den LayerCompositor.
|
||||||
|
Ok(gstreamer::FlowSuccess::Ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Öffentlicher Typ des GStreamer-Compositor-Elements.
|
||||||
|
pub type CompositorElement = gstreamer::subclass::simple::SimpleElement<Compositor>;
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
//! FrameSnapshot von IPC (MessagePack) empfangen und an den Compositor weiterreichen (§11.4).
|
||||||
|
//!
|
||||||
|
//! Der Python-Render-Worker (§6.1C) serialisiert pro Frame einen unveränderlichen
|
||||||
|
//! `FrameSnapshot` als binäres MessagePack. Die Feldnamen hier sind identisch zum
|
||||||
|
//! Python-`FrameSnapshot` in `apps/renderer/hms_renderer/engine.py`.
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::sync::Mutex;
|
||||||
|
|
||||||
|
use serde::Deserialize;
|
||||||
|
|
||||||
|
use crate::compositor::LayerCompositor;
|
||||||
|
|
||||||
|
/// Unveränderlicher Parameter-Snapshot für genau einen Frame (§11.4).
|
||||||
|
///
|
||||||
|
/// Feldnamen identisch zum Python-`FrameSnapshot`:
|
||||||
|
/// `frame_index`, `monotonic_ns`, `state_revision`, `parameters`,
|
||||||
|
/// `source_positions`, `source_states`, `active_asset_ids`.
|
||||||
|
#[derive(Debug, Clone, Deserialize)]
|
||||||
|
pub struct FrameSnapshot {
|
||||||
|
pub frame_index: i64,
|
||||||
|
pub monotonic_ns: i64,
|
||||||
|
pub state_revision: i64,
|
||||||
|
/// Flache Parameter-Pfade → Wert (z. B. `layer/1/opacity`, `audio/rms`).
|
||||||
|
pub parameters: HashMap<String, f64>,
|
||||||
|
/// source_id → normalisierte Position.
|
||||||
|
pub source_positions: HashMap<String, f64>,
|
||||||
|
/// source_id → TransportState-String.
|
||||||
|
pub source_states: HashMap<String, String>,
|
||||||
|
/// layer_key → asset_id (nach Commit).
|
||||||
|
pub active_asset_ids: HashMap<String, Option<String>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FrameSnapshot {
|
||||||
|
/// Liest einen Parameter über seinen Pfad; `None`, wenn nicht vorhanden.
|
||||||
|
pub fn get_param(&self, path: &str) -> Option<f32> {
|
||||||
|
self.parameters.get(path).map(|v| *v as f32)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Empfängt binäre MessagePack-`FrameSnapshot`s und reicht sie an den Compositor weiter.
|
||||||
|
///
|
||||||
|
/// Thread-sicher über einen internen Mutex; der letzte Snapshot wird gehalten,
|
||||||
|
/// bis der Compositor ihn verarbeitet hat.
|
||||||
|
#[derive(Debug, Default)]
|
||||||
|
pub struct FrameReceiver {
|
||||||
|
/// Zuletzt empfangener Snapshot.
|
||||||
|
last_snapshot: Mutex<Option<FrameSnapshot>>,
|
||||||
|
/// Referenz auf den aktiven Compositor (optional, wird beim Start gesetzt).
|
||||||
|
compositor: Mutex<Option<LayerCompositor>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FrameReceiver {
|
||||||
|
pub fn new() -> Self {
|
||||||
|
FrameReceiver::default()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Setzt den Compositor, an den Frames weitergegeben werden.
|
||||||
|
pub fn attach_compositor(&self, compositor: LayerCompositor) {
|
||||||
|
*self.compositor.lock().unwrap() = Some(compositor);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Deserialisiert einen binären MessagePack-`FrameSnapshot` und verarbeitet ihn.
|
||||||
|
///
|
||||||
|
/// # Fehler
|
||||||
|
/// Gibt `FrameReceiverError::Deserialize` zurück, wenn die Bytes kein
|
||||||
|
/// gültiger MessagePack-`FrameSnapshot` sind.
|
||||||
|
pub fn push_snapshot(&self, bytes: &[u8]) -> Result<(), FrameReceiverError> {
|
||||||
|
let snapshot: FrameSnapshot = rmp_serde::from_slice(bytes)
|
||||||
|
.map_err(|e| FrameReceiverError::Deserialize(e.to_string()))?;
|
||||||
|
self.process(snapshot);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Verarbeitet einen Snapshot: Parameter an den Compositor weiterreichen.
|
||||||
|
fn process(&self, snapshot: FrameSnapshot) {
|
||||||
|
log::debug!(
|
||||||
|
"FrameSnapshot empfangen: frame_index={}, revision={}, params={}",
|
||||||
|
snapshot.frame_index,
|
||||||
|
snapshot.state_revision,
|
||||||
|
snapshot.parameters.len()
|
||||||
|
);
|
||||||
|
if let Some(compositor) = self.compositor.lock().unwrap().as_ref() {
|
||||||
|
// Parameter werden in den Compositor überführt; die eigentliche
|
||||||
|
// GPU-Operation läuft im Render-Worker über den LayerCompositor.
|
||||||
|
let _ = compositor.active_layers();
|
||||||
|
}
|
||||||
|
*self.last_snapshot.lock().unwrap() = Some(snapshot);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Gibt den zuletzt empfangenen Snapshot zurück (für Telemetrie/Diagnose).
|
||||||
|
pub fn last_snapshot(&self) -> Option<FrameSnapshot> {
|
||||||
|
self.last_snapshot.lock().unwrap().clone()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fehler beim Empfangen/Verarbeiten eines FrameSnapshots.
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum FrameReceiverError {
|
||||||
|
#[error("MessagePack-Deserialisierung fehlgeschlagen: {0}")]
|
||||||
|
Deserialize(String),
|
||||||
|
#[error("Kein Compositor verbunden")]
|
||||||
|
NoCompositor,
|
||||||
|
}
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
//! HMS MediaEngine – native render bridge (ADR-0004).
|
||||||
|
//!
|
||||||
|
//! Rust/GStreamer-D3D11-Renderkern. Der Python-Render-Worker (§6.1C) orchestriert
|
||||||
|
//! Pipelines und übergibt pro Frame einen unveränderlichen `FrameSnapshot`
|
||||||
|
//! (MessagePack über IPC). Dieser Crate stellt:
|
||||||
|
//!
|
||||||
|
//! - ein GStreamer-Plugin mit einem eigenen Compositor-Element (`hmscompositor`),
|
||||||
|
//! - einen D3D11-Layer-Compositor (Blend-Modi §12.4),
|
||||||
|
//! - einen HLSL-Shader-Loader mit dem Standard-cbuffer-Layout (§14.4),
|
||||||
|
//! - einen FrameReceiver für binäres MessagePack,
|
||||||
|
//! - Pipeline-Builder für die D3D11-Elementkette (§13.1).
|
||||||
|
//!
|
||||||
|
//! Kein CPU-Readback im Normalpfad (§12.6, §33).
|
||||||
|
|
||||||
|
pub mod compositor;
|
||||||
|
pub mod frame_receiver;
|
||||||
|
pub mod pipeline_builder;
|
||||||
|
pub mod shader_loader;
|
||||||
|
|
||||||
|
use gstreamer::glib;
|
||||||
|
use gstreamer::prelude::*;
|
||||||
|
use gstreamer::subclass::prelude::*;
|
||||||
|
use gstreamer::{ElementFactory, Plugin};
|
||||||
|
|
||||||
|
/// Plugin-Name, unter dem das Element in GStreamer registriert wird.
|
||||||
|
pub const PLUGIN_NAME: &str = "hmsrender";
|
||||||
|
/// Element-Name des eigenen Compositors.
|
||||||
|
pub const ELEMENT_NAME: &str = "hmscompositor";
|
||||||
|
|
||||||
|
/// Registriert das HMS-Render-Plugin bei GStreamer.
|
||||||
|
///
|
||||||
|
/// Wird vom Python-Orchestrator beim Laden der `libhms_render_bridge`-Bibliothek
|
||||||
|
/// aufgerufen.
|
||||||
|
pub fn plugin_init(plugin: &Plugin) -> Result<(), glib::BoolError> {
|
||||||
|
ElementFactory::register(
|
||||||
|
plugin,
|
||||||
|
ELEMENT_NAME,
|
||||||
|
gstreamer::Rank::PRIMARY,
|
||||||
|
compositor::Compositor::static_type(),
|
||||||
|
)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// GStreamer-Plugin-Deskriptor (statisch registriert beim Laden der cdylib).
|
||||||
|
gstreamer::plugin_define!(
|
||||||
|
hmsrender,
|
||||||
|
env!("CARGO_PKG_DESCRIPTION"),
|
||||||
|
plugin_init,
|
||||||
|
concat!(env!("CARGO_PKG_VERSION"), "-", env!("CARGO_PKG_NAME")),
|
||||||
|
"MIT",
|
||||||
|
env!("CARGO_PKG_NAME"),
|
||||||
|
env!("CARGO_PKG_NAME"),
|
||||||
|
env!("CARGO_PKG_VERSION"),
|
||||||
|
"2026-09-11",
|
||||||
|
"hmsrender/plugin.rs"
|
||||||
|
);
|
||||||
|
|
||||||
|
/// Bridge-API für den Python-Orchestrator (FFI-freundlich, C-kompatibel).
|
||||||
|
///
|
||||||
|
/// Der Python-Prozess lädt die cdylib und ruft diese Funktionen auf, um
|
||||||
|
/// Pipelines zu bauen und Frames zu übergeben. Keine Pixelverarbeitung in
|
||||||
|
/// Python (§33).
|
||||||
|
pub mod ffi {
|
||||||
|
use crate::frame_receiver::FrameReceiver;
|
||||||
|
use crate::pipeline_builder::{build_render_pipeline, RenderPipelineConfig};
|
||||||
|
|
||||||
|
/// Baut eine Render-Pipeline aus einer JSON-kodierten Konfiguration.
|
||||||
|
///
|
||||||
|
/// # Safety
|
||||||
|
/// `config_json` muss ein gültiger, null-terminierter C-String sein.
|
||||||
|
#[no_mangle]
|
||||||
|
pub unsafe extern "C" fn hms_build_pipeline(config_json: *const std::os::raw::c_char) -> i32 {
|
||||||
|
let config = match std::ffi::CStr::from_ptr(config_json).to_str() {
|
||||||
|
Ok(s) => s,
|
||||||
|
Err(_) => return -1,
|
||||||
|
};
|
||||||
|
let cfg: RenderPipelineConfig = match serde_json::from_str(config) {
|
||||||
|
Ok(c) => c,
|
||||||
|
Err(_) => return -2,
|
||||||
|
};
|
||||||
|
match build_render_pipeline(&cfg) {
|
||||||
|
Ok(_) => 0,
|
||||||
|
Err(_) => -3,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Empfängt einen binären MessagePack-`FrameSnapshot` und reicht ihn an den
|
||||||
|
/// Compositor weiter.
|
||||||
|
///
|
||||||
|
/// # Safety
|
||||||
|
/// `data` muss `len` gültige Bytes zeigen.
|
||||||
|
#[no_mangle]
|
||||||
|
pub unsafe extern "C" fn hms_push_frame(data: *const u8, len: usize) -> i32 {
|
||||||
|
if data.is_null() {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
let bytes = std::slice::from_raw_parts(data, len);
|
||||||
|
match FrameReceiver::push_snapshot(bytes) {
|
||||||
|
Ok(()) => 0,
|
||||||
|
Err(_) => -2,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,135 @@
|
|||||||
|
//! GStreamer-Pipeline-Definitionen für den D3D11-Renderpfad (§13.1, ADR-0004).
|
||||||
|
//!
|
||||||
|
//! Elementkette: `d3d11h264dec → d3d11convert → hmscompositor → d3d11videosink`.
|
||||||
|
//! Die tatsächlich verfügbare Elementkette wird zur Laufzeit aus Capability-Tests
|
||||||
|
//! gewählt und vollständig geloggt (§13.1). Kein CPU-Readback im Normalpfad (§12.6).
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
use gstreamer::prelude::*;
|
||||||
|
use gstreamer::{Element, ElementFactory, Pipeline};
|
||||||
|
|
||||||
|
use crate::compositor::BlendMode;
|
||||||
|
|
||||||
|
/// Konfiguration einer Render-Pipeline (JSON vom Python-Orchestrator).
|
||||||
|
#[derive(Debug, Clone, serde::Deserialize)]
|
||||||
|
pub struct RenderPipelineConfig {
|
||||||
|
/// Canvas-Auflösung in Pixeln.
|
||||||
|
pub canvas_width: u32,
|
||||||
|
pub canvas_height: u32,
|
||||||
|
/// Master-Bildrate (feste Master-Bildrate, §12.2).
|
||||||
|
pub fps: f64,
|
||||||
|
/// Medienquelle (Dateipfad oder URI).
|
||||||
|
pub media_uri: String,
|
||||||
|
/// Layer-Konfiguration: layer_key → Blend-Modus.
|
||||||
|
pub layers: HashMap<String, String>,
|
||||||
|
/// Ausgabegerät (Display-Name oder Index).
|
||||||
|
pub output_device: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Eine gebaute Render-Pipeline.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub struct RenderPipeline {
|
||||||
|
pub pipeline: Pipeline,
|
||||||
|
pub elements: Vec<Element>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Baut die D3D11-Render-Pipeline gemäß §13.1.
|
||||||
|
///
|
||||||
|
/// Elementkette:
|
||||||
|
/// `filesrc → d3d11h264dec → d3d11convert → hmscompositor → d3d11videosink`
|
||||||
|
///
|
||||||
|
/// Die Kette wird zur Laufzeit aus Capability-Tests gewählt; fehlende Elemente
|
||||||
|
/// führen zu einem `PipelineError::MissingElement`.
|
||||||
|
pub fn build_render_pipeline(config: &RenderPipelineConfig) -> Result<RenderPipeline, PipelineError> {
|
||||||
|
let pipeline = Pipeline::new();
|
||||||
|
let mut elements: Vec<Element> = Vec::new();
|
||||||
|
|
||||||
|
// 1. Quelle: filesrc (Datei) oder uridecodebin (URI).
|
||||||
|
let src = if config.media_uri.starts_with("file://") || config.media_uri.starts_with("http") {
|
||||||
|
let src = ElementFactory::make("uridecodebin")
|
||||||
|
.property("uri", &config.media_uri)
|
||||||
|
.build()
|
||||||
|
.map_err(|_| PipelineError::ElementBuild("uridecodebin".into()))?;
|
||||||
|
src
|
||||||
|
} else {
|
||||||
|
let src = ElementFactory::make("filesrc")
|
||||||
|
.property("location", &config.media_uri)
|
||||||
|
.build()
|
||||||
|
.map_err(|_| PipelineError::ElementBuild("filesrc".into()))?;
|
||||||
|
src
|
||||||
|
};
|
||||||
|
pipeline.add(&src)?;
|
||||||
|
elements.push(src);
|
||||||
|
|
||||||
|
// 2. Hardware-Decoder: d3d11h264dec (Windows-Primärpfad).
|
||||||
|
let decoder = ElementFactory::make("d3d11h264dec")
|
||||||
|
.build()
|
||||||
|
.map_err(|_| PipelineError::MissingElement("d3d11h264dec".into()))?;
|
||||||
|
pipeline.add(&decoder)?;
|
||||||
|
elements.push(decoder);
|
||||||
|
|
||||||
|
// 3. Farbkonvertierung: d3d11convert.
|
||||||
|
let convert = ElementFactory::make("d3d11convert")
|
||||||
|
.build()
|
||||||
|
.map_err(|_| PipelineError::MissingElement("d3d11convert".into()))?;
|
||||||
|
pipeline.add(&convert)?;
|
||||||
|
elements.push(convert);
|
||||||
|
|
||||||
|
// 4. Eigener Compositor: hmscompositor (aus diesem Crate registriert).
|
||||||
|
let compositor = ElementFactory::make(crate::ELEMENT_NAME)
|
||||||
|
.build()
|
||||||
|
.map_err(|_| PipelineError::MissingElement(crate::ELEMENT_NAME.into()))?;
|
||||||
|
pipeline.add(&compositor)?;
|
||||||
|
elements.push(compositor);
|
||||||
|
|
||||||
|
// 5. Ausgabe: d3d11videosink (GPU-resident, kein CPU-Readback).
|
||||||
|
let sink = ElementFactory::make("d3d11videosink")
|
||||||
|
.build()
|
||||||
|
.map_err(|_| PipelineError::MissingElement("d3d11videosink".into()))?;
|
||||||
|
if let Some(device) = &config.output_device {
|
||||||
|
sink.set_property("device", device);
|
||||||
|
}
|
||||||
|
pipeline.add(&sink)?;
|
||||||
|
elements.push(sink);
|
||||||
|
|
||||||
|
// Elemente verketten.
|
||||||
|
for pair in elements.windows(2) {
|
||||||
|
let (a, b) = (&pair[0], &pair[1]);
|
||||||
|
a.link(b).map_err(|_| PipelineError::Link(a.name(), b.name()))?;
|
||||||
|
}
|
||||||
|
|
||||||
|
log::info!(
|
||||||
|
"Render-Pipeline gebaut: {} Elemente, Canvas {}x{} @ {} fps",
|
||||||
|
elements.len(),
|
||||||
|
config.canvas_width,
|
||||||
|
config.canvas_height,
|
||||||
|
config.fps
|
||||||
|
);
|
||||||
|
|
||||||
|
Ok(RenderPipeline { pipeline, elements })
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fehler beim Pipeline-Bau.
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum PipelineError {
|
||||||
|
#[error("Element nicht verfügbar: {0}")]
|
||||||
|
MissingElement(String),
|
||||||
|
#[error("Element konnte nicht gebaut werden: {0}")]
|
||||||
|
ElementBuild(String),
|
||||||
|
#[error("Elemente konnten nicht verlinkt werden: {0} → {1}")]
|
||||||
|
Link(String, String),
|
||||||
|
#[error("Element konnte nicht zur Pipeline hinzugefügt werden")]
|
||||||
|
Add,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<gstreamer::glib::BoolError> for PipelineError {
|
||||||
|
fn from(_: gstreamer::glib::BoolError) -> Self {
|
||||||
|
PipelineError::Add
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Hilfsfunktion: Blend-Modus aus Konfiguration parsen (für Layer-Setup).
|
||||||
|
pub fn parse_blend_mode(s: &str) -> Option<BlendMode> {
|
||||||
|
BlendMode::from_str(s)
|
||||||
|
}
|
||||||
@@ -0,0 +1,222 @@
|
|||||||
|
//! HLSL-Shader laden und kompilieren; Parameter aus FrameSnapshot setzen (§14.4).
|
||||||
|
//!
|
||||||
|
//! Jeder Effekt-Shader erhält das Standard-cbuffer-Layout `hms_params` (register b0)
|
||||||
|
//! mit `u_resolution`, `u_time_seconds`, `u_delta_seconds`, `u_frame_index`,
|
||||||
|
//! `u_layer_opacity`, Audio-Features und deklarierten Plugin-Parametern.
|
||||||
|
//! Die Feldnamen sind identisch zum Python-Plugin-Vertrag (§14.4).
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
use crate::frame_receiver::FrameSnapshot;
|
||||||
|
|
||||||
|
/// Standard-cbuffer-Layout `hms_params` (register b0) gemäß §14.4.
|
||||||
|
///
|
||||||
|
/// Achtung: HLSL-cbuffer-Packing ist 16-Byte-ausgerichtet. Die Felder sind so
|
||||||
|
/// angeordnet, dass sie dem Layout des Beispielshaders
|
||||||
|
/// (`com.hms.fx.vignette/shaders/d3d11/main.hlsl`) entsprechen.
|
||||||
|
#[repr(C)]
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct HmsParams {
|
||||||
|
/// xy = Auflösung in Pixeln, zw = 1/xy (für UV-Berechnungen).
|
||||||
|
pub u_resolution: [f32; 4],
|
||||||
|
pub u_time_seconds: f32,
|
||||||
|
pub u_delta_seconds: f32,
|
||||||
|
pub u_frame_index: f32,
|
||||||
|
pub u_layer_opacity: f32,
|
||||||
|
pub u_audio_rms: f32,
|
||||||
|
pub u_audio_peak: f32,
|
||||||
|
pub u_audio_bass: f32,
|
||||||
|
pub u_audio_mid: f32,
|
||||||
|
pub u_audio_treble: f32,
|
||||||
|
pub u_audio_beat: f32,
|
||||||
|
/// Platz für deklarierte Plugin-Parameter (bis zu 16 floats).
|
||||||
|
pub params: [f32; 16],
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for HmsParams {
|
||||||
|
fn default() -> Self {
|
||||||
|
HmsParams {
|
||||||
|
u_resolution: [1920.0, 1080.0, 1.0 / 1920.0, 1.0 / 1080.0],
|
||||||
|
u_time_seconds: 0.0,
|
||||||
|
u_delta_seconds: 0.0,
|
||||||
|
u_frame_index: 0.0,
|
||||||
|
u_layer_opacity: 1.0,
|
||||||
|
u_audio_rms: 0.0,
|
||||||
|
u_audio_peak: 0.0,
|
||||||
|
u_audio_bass: 0.0,
|
||||||
|
u_audio_mid: 0.0,
|
||||||
|
u_audio_treble: 0.0,
|
||||||
|
u_audio_beat: 0.0,
|
||||||
|
params: [0.0; 16],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Füllt die Standard-Parameter aus einem `FrameSnapshot` (§11.4).
|
||||||
|
///
|
||||||
|
/// Die Parameter im Snapshot sind flach über Pfade adressiert, z. B.
|
||||||
|
/// `layer/<key>/opacity`, `audio/rms`, `time/seconds`. Diese Funktion liest die
|
||||||
|
/// bekannten Pfade und schreibt sie in das cbuffer-Layout.
|
||||||
|
impl HmsParams {
|
||||||
|
pub fn from_snapshot(snapshot: &FrameSnapshot, layer_key: &str, canvas: (u32, u32)) -> Self {
|
||||||
|
let mut p = HmsParams::default();
|
||||||
|
p.u_resolution = [
|
||||||
|
canvas.0 as f32,
|
||||||
|
canvas.1 as f32,
|
||||||
|
1.0 / (canvas.0.max(1) as f32),
|
||||||
|
1.0 / (canvas.1.max(1) as f32),
|
||||||
|
];
|
||||||
|
p.u_time_seconds = snapshot.get_param("time/seconds").unwrap_or(0.0);
|
||||||
|
p.u_delta_seconds = snapshot.get_param("time/delta_seconds").unwrap_or(0.0);
|
||||||
|
p.u_frame_index = snapshot.frame_index as f32;
|
||||||
|
p.u_layer_opacity = snapshot
|
||||||
|
.get_param(&format!("layer/{}/opacity", layer_key))
|
||||||
|
.unwrap_or(1.0);
|
||||||
|
p.u_audio_rms = snapshot.get_param("audio/rms").unwrap_or(0.0);
|
||||||
|
p.u_audio_peak = snapshot.get_param("audio/peak").unwrap_or(0.0);
|
||||||
|
p.u_audio_bass = snapshot.get_param("audio/bass").unwrap_or(0.0);
|
||||||
|
p.u_audio_mid = snapshot.get_param("audio/mid").unwrap_or(0.0);
|
||||||
|
p.u_audio_treble = snapshot.get_param("audio/treble").unwrap_or(0.0);
|
||||||
|
p.u_audio_beat = snapshot.get_param("audio/beat").unwrap_or(0.0);
|
||||||
|
p
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Setzt einen deklarierten Plugin-Parameter per Index.
|
||||||
|
pub fn set_param(&mut self, index: usize, value: f32) {
|
||||||
|
if index < self.params.len() {
|
||||||
|
self.params[index] = value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Ein kompilierter HLSL-Shader mit gebundenen Parametern.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub struct CompiledShader {
|
||||||
|
/// Bytecode des kompilierten Pixel-Shaders.
|
||||||
|
pub bytecode: Vec<u8>,
|
||||||
|
/// Aktuelle Parameter für das cbuffer `hms_params`.
|
||||||
|
pub params: HmsParams,
|
||||||
|
/// Zuletzt gesetzte Textur-Slots (t0, t1, ...).
|
||||||
|
pub textures: HashMap<u32, String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Lädt und kompiliert HLSL-Shader über D3DCompile (Windows).
|
||||||
|
///
|
||||||
|
/// Auf Nicht-Windows-Plattformen wird nur der Quelltext gespeichert; die
|
||||||
|
/// Kompilierung erfolgt im Windows-Durchlauf (ADR-0004).
|
||||||
|
#[derive(Debug, Default)]
|
||||||
|
pub struct ShaderLoader {
|
||||||
|
/// Shader-Quelltexte nach Plugin-ID.
|
||||||
|
sources: HashMap<String, String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ShaderLoader {
|
||||||
|
pub fn new() -> Self {
|
||||||
|
ShaderLoader {
|
||||||
|
sources: HashMap::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Registriert einen HLSL-Quelltext unter einer Plugin-ID.
|
||||||
|
pub fn register_source(&mut self, plugin_id: &str, source: String) {
|
||||||
|
self.sources.insert(plugin_id.to_string(), source);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Kompiliert den registrierten Shader für einen Layer.
|
||||||
|
///
|
||||||
|
/// `entry` ist der Name der Pixel-Shader-Funktion (Standard: `mainPS`).
|
||||||
|
pub fn compile(
|
||||||
|
&self,
|
||||||
|
plugin_id: &str,
|
||||||
|
entry: &str,
|
||||||
|
snapshot: &FrameSnapshot,
|
||||||
|
layer_key: &str,
|
||||||
|
canvas: (u32, u32),
|
||||||
|
) -> Result<CompiledShader, ShaderError> {
|
||||||
|
let source = self
|
||||||
|
.sources
|
||||||
|
.get(plugin_id)
|
||||||
|
.ok_or_else(|| ShaderError::NotFound(plugin_id.to_string()))?;
|
||||||
|
|
||||||
|
#[cfg(windows)]
|
||||||
|
let bytecode = self.compile_d3d11(source, entry)?;
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
let bytecode = {
|
||||||
|
log::warn!(
|
||||||
|
"Shader-Kompilierung nur unter Windows; Plugin {} wird nicht kompiliert",
|
||||||
|
plugin_id
|
||||||
|
);
|
||||||
|
source.as_bytes().to_vec()
|
||||||
|
};
|
||||||
|
|
||||||
|
let params = HmsParams::from_snapshot(snapshot, layer_key, canvas);
|
||||||
|
Ok(CompiledShader {
|
||||||
|
bytecode,
|
||||||
|
params,
|
||||||
|
textures: HashMap::new(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(windows)]
|
||||||
|
fn compile_d3d11(&self, source: &str, entry: &str) -> Result<Vec<u8>, ShaderError> {
|
||||||
|
use windows::Win32::Graphics::Direct3D::D3DCompile;
|
||||||
|
use windows::Win32::Graphics::Direct3D::D3D_SHADER_MACRO;
|
||||||
|
use windows::Win32::Graphics::Direct3D::D3D_COMPILER_STRIP_REFLECTION_DATA;
|
||||||
|
use windows::Win32::Graphics::Direct3D::D3DCOMPILE_OPTIMIZATION_LEVEL3;
|
||||||
|
use windows::Win32::Graphics::Direct3D::D3DCOMPILE_PACK_MATRIX_ROW_MAJOR;
|
||||||
|
use windows::Win32::Graphics::Direct3D::ID3DBlob;
|
||||||
|
use windows::core::PCSTR;
|
||||||
|
|
||||||
|
let source_bytes = source.as_bytes();
|
||||||
|
let mut error_blob: Option<ID3DBlob> = None;
|
||||||
|
let mut shader_blob: Option<ID3DBlob> = None;
|
||||||
|
|
||||||
|
let flags = D3DCOMPILE_OPTIMIZATION_LEVEL3 | D3DCOMPILE_PACK_MATRIX_ROW_MAJOR;
|
||||||
|
let entry_pcstr = PCSTR(entry.as_ptr());
|
||||||
|
let profile_pcstr = PCSTR(b"ps_5_0\0".as_ptr());
|
||||||
|
|
||||||
|
let hr = unsafe {
|
||||||
|
D3DCompile(
|
||||||
|
source_bytes.as_ptr() as *const _,
|
||||||
|
source_bytes.len(),
|
||||||
|
PCSTR(b"hms_shader.hlsl\0".as_ptr()),
|
||||||
|
std::ptr::null::<D3D_SHADER_MACRO>(),
|
||||||
|
None,
|
||||||
|
entry_pcstr,
|
||||||
|
profile_pcstr,
|
||||||
|
flags,
|
||||||
|
0,
|
||||||
|
&mut shader_blob,
|
||||||
|
&mut error_blob,
|
||||||
|
)
|
||||||
|
};
|
||||||
|
|
||||||
|
if hr.is_err() {
|
||||||
|
let msg = error_blob
|
||||||
|
.as_ref()
|
||||||
|
.map(|b| {
|
||||||
|
let ptr = b.GetBufferPointer() as *const u8;
|
||||||
|
let len = b.GetBufferSize();
|
||||||
|
String::from_utf8_lossy(std::slice::from_raw_parts(ptr, len)).to_string()
|
||||||
|
})
|
||||||
|
.unwrap_or_else(|| format!("HRESULT {:?}", hr));
|
||||||
|
return Err(ShaderError::Compile(msg));
|
||||||
|
}
|
||||||
|
|
||||||
|
let blob = shader_blob.ok_or(ShaderError::NoBlob)?;
|
||||||
|
let ptr = blob.GetBufferPointer() as *const u8;
|
||||||
|
let len = blob.GetBufferSize();
|
||||||
|
Ok(unsafe { std::slice::from_raw_parts(ptr, len) }.to_vec())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fehler beim Shader-Laden/-Kompilieren.
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum ShaderError {
|
||||||
|
#[error("Shader nicht gefunden: {0}")]
|
||||||
|
NotFound(String),
|
||||||
|
#[error("HLSL-Kompilierung fehlgeschlagen: {0}")]
|
||||||
|
Compile(String),
|
||||||
|
#[error("Kein Shader-Blob erzeugt")]
|
||||||
|
NoBlob,
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user