CV-Project

3D Gaussian Splatting is a technique used in computer vision and graphics for rendering 3D scenes. It represents 3D objects as Gaussian distributions in space, optimized via photometric loss. It has shown:

  • High efficiency and real-time rendering capabilities.
  • Advantages over neural network-based methods like NeRF due to simpler representation and rasterization compatibility.

Mip-Splatting Innovation

Mip-Splatting enhances 3D Gaussian Splatting by introducing:

  1. 3D Smoothing Filters: To constrain the Gaussian primitives’ frequencies, ensuring adherence to sampling constraints and reducing high-frequency artifacts during zoom-in.

  2. 2D Mip Filters: Aiming to mitigate aliasing when zooming out by simulating a box filter from physical imaging processes.

Slide 1: Introduction

Title: Mip-Splatting: Improvements for 3D Gaussian Splatting
Key Points:

  • What is 3DGS?
    • Represents 3D scenes using Gaussian primitives (position, size, orientation).
    • Efficient real-time rendering but suffers from artifacts when scaling views.
  • Problems in 3DGS:
    • High-frequency artifacts during zoom-in (thinning edges, erosion).
    • Brightness and dilation artifacts during zoom-out (over-smoothed or distorted details).
    • Poor generalization to new scales and resolutions.

Slide 2: Mip-Splatting Solutions

Title: How Mip-Splatting Solves 3DGS Issues

  1. 3D Smoothing Filter

    • Applies low-pass filtering to Gaussian primitives in 3D space.
    • Constrains maximum frequencies to avoid high-frequency artifacts.
    • Effect: Eliminates thinning edges and erosion when zooming in.

  2. 2D Mip Filter

    • Replaces 3DGS's dilation operation with a box-filter approximation.
    • Simulates the imaging process for anti-aliasing.
    • Effect: Prevents brightness/dilation artifacts during zoom-out.

Slide 3: Results and Impact

Title: Results of Mip-Splatting

  1. Improved Rendering Quality

    • Consistent image fidelity across resolutions and scales.
    • Handles unseen sampling rates effectively (zoom-in/zoom-out scenarios).

  2. Generalization Across Scales

    • Single-scale training achieves multi-scale rendering with no added overhead.

  3. Efficiency

    • Dynamic Gaussian parameter adjustment reduces memory overhead.

Visuals:
Include one comparison figure (e.g., a zoom-in/zoom-out comparison showing reduced artifacts in Mip-Splatting).

When Change Resolution

Zoom in:
里的进的时候，辐条很细

Zoom Out:
离的远的时候，车轮膨胀

3D Smoothing Filter (Brief Overview)

Purpose:
Eliminates high-frequency artifacts (e.g., thinning edges) during zoom-in by constraining the frequencies of 3D Gaussian primitives.

Key Equation:
The smoothed Gaussian is defined as:

Where:

  • : Maximum sampling frequency of Gaussian , computed from training views.
  • : Scalar controlling the filter size.
  • : Identity matrix.

Steps:

  1. Compute : Use Nyquist limit based on camera focal length and depth.
  2. Apply Filter: Add a low-pass filter to the Gaussian covariance matrix.

Effect:
Prevents high-frequency artifacts, ensuring smooth rendering when zooming in on 3D scenes.

2D Mip Filter (Brief Overview)

Purpose:
Mitigates aliasing and brightness/dilation artifacts during zoom-out by simulating the physical imaging process with a box filter approximation.

Key Equation:
The filtered 2D Gaussian in screen space is defined as:

Where:

  • : 2D covariance matrix of the projected Gaussian.
  • : Scalar controlling the filter size, chosen to match the pixel size.
  • : 2D identity matrix.

Steps:

  1. Replace Dilation:
    Replace 3DGS’s screen-space dilation with this 2D Gaussian filter that approximates a box filter.
  2. Adjust Filter Size:
    Set ss to cover a single pixel in screen space for consistent anti-aliasing.

Effect:

  • Prevents brightness/dilation artifacts during zoom-out.
  • Ensures smooth and natural rendering across scales with reduced aliasing.

Would you like more visual examples or implementation guidance?

Table Of Contents
CV-Project
Slide 1: Introduction
Slide 2: Mip-Splatting Solutions
Slide 3: Results and Impact
3D Smoothing Filter (Brief Overview)
2D Mip Filter (Brief Overview)