mediamtx/internal/hls/muxer_variant_fmp4_part.go

104 lines
2.1 KiB
Go
Raw Normal View History

Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
package hls
import (
"bytes"
"io"
"strconv"
"time"
"github.com/aler9/gortsplib"
2022-08-05 21:50:45 +00:00
"github.com/aler9/gortsplib/pkg/mpeg4audio"
2022-06-04 14:51:57 +00:00
"github.com/aler9/rtsp-simple-server/internal/hls/fmp4"
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
)
func fmp4PartName(id uint64) string {
return "part" + strconv.FormatUint(id, 10)
}
type muxerVariantFMP4Part struct {
videoTrack *gortsplib.TrackH264
2022-08-05 21:50:45 +00:00
audioTrack *gortsplib.TrackMPEG4Audio
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
id uint64
isIndependent bool
videoSamples []*fmp4.VideoSample
audioSamples []*fmp4.AudioSample
content []byte
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
renderedDuration time.Duration
}
func newMuxerVariantFMP4Part(
videoTrack *gortsplib.TrackH264,
2022-08-05 21:50:45 +00:00
audioTrack *gortsplib.TrackMPEG4Audio,
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
id uint64,
) *muxerVariantFMP4Part {
p := &muxerVariantFMP4Part{
videoTrack: videoTrack,
audioTrack: audioTrack,
id: id,
}
if videoTrack == nil {
p.isIndependent = true
}
return p
}
func (p *muxerVariantFMP4Part) name() string {
return fmp4PartName(p.id)
}
func (p *muxerVariantFMP4Part) reader() io.Reader {
return bytes.NewReader(p.content)
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
}
func (p *muxerVariantFMP4Part) duration() time.Duration {
if p.videoTrack != nil {
ret := time.Duration(0)
for _, e := range p.videoSamples {
ret += e.Duration()
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
}
return ret
}
// use the sum of the default duration of all samples,
// not the real duration,
// otherwise on iPhone iOS the stream freezes.
return time.Duration(len(p.audioSamples)) * time.Second *
2022-08-05 21:50:45 +00:00
time.Duration(mpeg4audio.SamplesPerAccessUnit) / time.Duration(p.audioTrack.ClockRate())
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
}
func (p *muxerVariantFMP4Part) finalize() error {
if len(p.videoSamples) > 0 || len(p.audioSamples) > 0 {
var err error
p.content, err = fmp4.GeneratePart(
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
p.videoTrack,
p.audioTrack,
p.videoSamples,
p.audioSamples)
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
if err != nil {
return err
}
p.renderedDuration = p.duration()
}
p.videoSamples = nil
p.audioSamples = nil
return nil
}
func (p *muxerVariantFMP4Part) writeH264(sample *fmp4.VideoSample) {
if sample.IDRPresent {
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
p.isIndependent = true
}
p.videoSamples = append(p.videoSamples, sample)
}
func (p *muxerVariantFMP4Part) writeAAC(sample *fmp4.AudioSample) {
Implement Low-Latency HLS (#938) * add hlsVariant parameter * hls: split muxer into variants * hls: implement fmp4 segments * hls muxer: implement low latency mode * hls muxer: support audio with fmp4 mode * hls muxer: rewrite file router * hls muxer: implement preload hint * hls muxer: add various error codes * hls muxer: use explicit flags * hls muxer: fix error in aac pts * hls muxer: fix sudden freezes with video+audio * hls muxer: skip empty parts * hls muxer: fix video FPS * hls muxer: add parameter hlsPartDuration * hls muxer: refactor fmp4 muxer * hls muxer: fix CAN-SKIP-UNTIL * hls muxer: refactor code * hls muxer: show only parts of last 2 segments * hls muxer: implementa playlist delta updates * hls muxer: change playlist content type * hls muxer: improve video dts precision * hls muxer: fix video sample flags * hls muxer: improve iphone audio support * hls muxer: improve mp4 timestamp precision * hls muxer: add offset between pts and dts * hls muxer: close muxer in case of error * hls muxer: stop logging requests with the info level * hls muxer: rename entry into sample * hls muxer: compensate video dts error over time * hls muxer: change default segment count * hls muxer: add starting gap * hls muxer: set default part duration to 200ms * hls muxer: fix audio-only streams on ios * hls muxer: add playsinline attribute to video tag of default web page * hls muxer: keep mpegts as the default hls variant * hls muxer: implement encryption * hls muxer: rewrite dts estimation * hls muxer: improve DTS precision * hls muxer: use right SPS/PPS for each sample * hls muxer: adjust part duration dynamically * add comments * update readme * hls muxer: fix memory leak * hls muxer: decrease ram consumption
2022-05-31 17:17:26 +00:00
p.audioSamples = append(p.audioSamples, sample)
}