[Ron Capik]

< ....snip.. >

If Radium doesn't take the trouble to understand your clear exposition,
he's beyond help.

Jerry
--

You sound like someone who has never followed a
"Radium" thread before. ;-}

[The extent of the cross-post lists says much about
his usenet ~mastery~ ]

Later...

Ron
--

 

[Rich Grise]

Sound is physical - Light is electromagnetic radiation

That would make a great Sci-Fi effect to depict 'beings' in a different
temporal dimension co-existing with us but what you describe is the effect
of motion linking the audio analogy to video which is not valid. Sound is an
air-pressure wave whose speed changes depending on the medium whereas light
is part of the electromagnetic spectrum whose speed is fixed to the speed of
light and except for some very high-end academic experiments never changes.
Never-the-less it's a good special effects used in a modified way in the BBC
production Ultra Violet.

I believe the point was about the Doppler effect, which they both
experience, albeit in very different media. In fact, you can see this
in water waves, if you have a boat that's going more slowly than the
"speed of wave" in that body of water.

But, yes, alas, other than moving at a significant fraction of c, I
don't think there's any way to exploit blue-shift. ;-)

Hope This Helps!
Rich

 

[Ron N.]

Anyways, Adobe Audition and voice-changers allow the frequencies of an
audio signal to be shifted w/out low-pass filtering or changing the
tempo. There are two video-equivalents of this because, while audio
has only one frequency component [temporal], video has two [temporal
and spatial].

Voice-changing and pitch shifting algorithms work
by duplicating or throwing away information which
the ear can rarely detect, but would be really
obvious to the eye (a phoneme may sound the same
with less or more excitation cycles, but a picture
of your family would not look the same with a some
people missing, or with twin children added.)

MPEG-2 video compression already does a coarse
equivalent of time-domain pitch shifting via motion
estimation and compensation, e.g. it throws away
whole frames of video and repeats the spacial
components from previous frames, sometimes skipping
some new motion (leading to jerky patches of video
playback if the compression rate is lower than a
suitable information bandwidth).


IMHO. YMMV.

 

[Jim Kelley]

The temporal video-equivalent would be changing the rate of back/
forth, up-down or other repetitive/cyclical movement [such as wing-
flapping or flickering of lights] of the video signal without high/low-
pass-filtering, separating any portion of the video signal, or
changing the speed at which the video-signal -- just as voice-changers
can lower the frequency of audio without changing the speed of the
audio. Using a voice-changer to decrease the pitch your voice will not
cause your speech to slow down.

In order to change the 'spacial frequency' aspect of video data
without altering the 'temporal' aspect you have to either add or
delete information interstitially and then play back the altered data
at a compensated data rate. (Using the term 'data' in the most
general sense here.)

jk

 

[isw]

MPEG-2 video compression already does a coarse
equivalent of time-domain pitch shifting via motion
estimation and compensation, e.g. it throws away
whole frames of video and repeats the spacial
components from previous frames, sometimes skipping
some new motion (leading to jerky patches of video
playback if the compression rate is lower than a
suitable information bandwidth).

Not really. MPEG-2 is frame rate conservative, end-to-end. In fact, the
output frame rate is required by the standard to be *identical* to the
input rate. That has to be the case for it to be able to handle NTSC or
PAL delivered to ordinary TV sets.

And if there are "jerky patches", that just means that it was improperly
applied.

Isaac

 

[Ron N.]

Not really. MPEG-2 is frame rate conservative, end-to-end. In fact, the
output frame rate is required by the standard to be *identical* to the
input rate. That has to be the case for it to be able to handle NTSC or
PAL delivered to ordinary TV sets.

MPEG is frame rate conservative, but only the I frames are
actually sent as full images. The P and B frames are made
up out of some duplicated and possibly displaced contents of
other frames, plus some quantized portion of an error vector
depending on the compression rate. Thus the data bandwidth
required for an P and B frames is a fraction of that typically
required for the full image, as contained a nearby I frame.

Some pitch-shifters or time-stretchers also duplicate and
blend preceding and following periods of waveforms or
spectral frame contents.

 

[isw]

MPEG is frame rate conservative, but only the I frames are
actually sent as full images. The P and B frames are made
up out of some duplicated and possibly displaced contents of
other frames, plus some quantized portion of an error vector
depending on the compression rate. Thus the data bandwidth
required for an P and B frames is a fraction of that typically
required for the full image, as contained a nearby I frame.

Yup. Also, some of those frames are sent out of sequence (I or P
"anchor" frames must be present first, in order for the interpolated
frames to be recreated), but every frame has a representation of some
sort in the stream, every frame gets put in its proper place by the
decoder, and no frames are skipped.

Some pitch-shifters or time-stretchers also duplicate and
blend preceding and following periods of waveforms or
spectral frame contents.

Yes again. The difference is that with MPEG video the "duplicating and
blending" has zero effect on the frame rate (i.e., the temporal
resolution).

There is an interesting sort-of exception to frame rate conservation,
when film source is encoded at 24 FPS (actually about 23.98) and the
decoder performs 3-2 pulldown to deliver the NTSC-required 29.97 FPS,
but that's not germane to this discussion.

Isaac

 

[Ron N.]

There is an interesting sort-of exception to frame rate conservation,
when film source is encoded at 24 FPS (actually about 23.98) and the
decoder performs 3-2 pulldown to deliver the NTSC-required 29.97 FPS,
but that's not germane to this discussion.

Actually, it is very germane, since 3-2 pulldown is similar
to how some primitive audio pitch/rate changing hardware worked,
by duplicating small time domain frames of audio at a fixed
proportion and rate. Some MPEG decoders do "special effects"
by varying the frame duplicate/drop fractions to slow down
or speed up playback using the same mechanism as for pulldown.

 

[robert bristow-johnson]

Some pitch-shifters or time-stretchers also duplicate and
blend preceding and following periods of waveforms or
spectral frame contents.

when i first saw the thread title, that's what i first thought about.
actually, not pitch-shifting but more time-scaling. it seems to me
natural that if they were speeding up or slowing down the motion in
the video (which means only for the termporal dimension, not either
"x" or "y"), that would naturally correspond to the same speeding up
or slowing down of tempo (without pitch change) of the audio. if you
twist the knob that makes the actress talk faster (Ms. Motormouth), it
shouldn't be upshifting her pitch to sound like Wendy or Bebe in South
Park.

r b-j

 

[robert bristow-johnson]

The fact that there is essentially no relation between these two
entities -- i.e. the data stream is comprised of a sequence of
descriptions of a series of still images -- is the reason why what you
want to do is almost certainly impossible.

If you really want to try, the first step will be to devise a method of
recording video that does not quantize the temporal axis; i.e. not using
a sequence of still images.

can't we think of the intensity (and chroma components) of a
particular point (x,y) of a still image as a sampled (at a rate of 30
Hz) value of a continuous-time signal that represents intensity at
that point? i.e. we have I(x,y,t) being sampled as I(x,y,n*T). and
then use some kinda interpolation to hypothetically reconstruct the
"still" images in between the sequence we are given? i imagine there
would be some blurring, but if the resolution was very good to start
with, would that not work. at least as a beginning point?

r b-j

 

[Gene E. Bloch]

can't we think of the intensity (and chroma components) of a
particular point (x,y) of a still image as a sampled (at a rate of 30
Hz) value of a continuous-time signal that represents intensity at
that point? i.e. we have I(x,y,t) being sampled as I(x,y,n*T). and
then use some kinda interpolation to hypothetically reconstruct the
"still" images in between the sequence we are given? i imagine there
would be some blurring, but if the resolution was very good to start
with, would that not work. at least as a beginning point?

r b-j

Yes. Radium included that idea in his first thread of the week, but it
was not liked by the community[1]

I thought it was perhaps his only cogent idea, but OTOH, I'm not sure
where to take it.

[1] At least in the part of the thread that I read.

 

[isw]

can't we think of the intensity (and chroma components) of a
particular point (x,y) of a still image as a sampled (at a rate of 30
Hz) value of a continuous-time signal that represents intensity at
that point? i.e. we have I(x,y,t) being sampled as I(x,y,n*T). and
then use some kinda interpolation to hypothetically reconstruct the
"still" images in between the sequence we are given? i imagine there
would be some blurring, but if the resolution was very good to start
with, would that not work. at least as a beginning point?

That's not far from the way PAL (25 FPS)-to-NTSC (29.97 FPS) converters
work.

Isaac

 

[Jerry Avins]

Actually, it is very germane, since 3-2 pulldown is similar
to how some primitive audio pitch/rate changing hardware worked,
by duplicating small time domain frames of audio at a fixed
proportion and rate. Some MPEG decoders do "special effects"
by varying the frame duplicate/drop fractions to slow down
or speed up playback using the same mechanism as for pulldown.

I think he means the change from 2:2 pull-down's natural rate of 30fps
and TV's 29.97. I call that negligible.

jerry

 

[Jerry Avins]

when i first saw the thread title, that's what i first thought about.
actually, not pitch-shifting but more time-scaling. it seems to me
natural that if they were speeding up or slowing down the motion in
the video (which means only for the termporal dimension, not either
"x" or "y"), that would naturally correspond to the same speeding up
or slowing down of tempo (without pitch change) of the audio. if you
twist the knob that makes the actress talk faster (Ms. Motormouth), it
shouldn't be upshifting her pitch to sound like Wendy or Bebe in South
Park.

But he wants her to talk faster, say the same number of words, and
finish in the same time! What's worse, I think he's serious.

Jerry

 

[Radium]

when i first saw the thread title, that's what i first thought about.
actually, not pitch-shifting but more time-scaling.

That's the opposite of what I'm looking for.

it seems to me
natural that if they were speeding up or slowing down the motion in
the video (which means only for the termporal dimension, not either
"x" or "y"), that would naturally correspond to the same speeding up
or slowing down of tempo (without pitch change) of the audio. if you
twist the knob that makes the actress talk faster (Ms. Motormouth), it
shouldn't be upshifting her pitch to sound like Wendy or Bebe in South
Park.

I want the actress to talk at the same speed, at a lower-pitch, and
finish at the same-time without any low-pass filtering.

 

[isw]

I think he means the change from 2:2 pull-down's natural rate of 30fps

30 fps? What is that the "natural rate" of? I'm not aware of anything
that runs "naturally" at 30 fps.

Isaac

 

[robert bristow-johnson]

That's the opposite of what I'm looking for.


I want the actress to talk at the same speed, at a lower-pitch, and
finish at the same-time without any low-pass filtering.

okay, so that is a real-time pitch shifter. you can buy those things
and you can buy plug-ins that do it. so what is the "video-
equivalent" to a real-time pitch shifter?

r b-j

(Jerry, i hope i didn't step in a puddle of dung, did i? were you
trying to warn me away?)

 

[Radium]

so what is the "video-
equivalent" to a real-time pitch shifter?

I wish I knew. This is so interesting for me yet so difficult for me
to answer.

What is the "video-equivalent" to a real-time pitch shifter if the
video is B&W? Since I've been giving wrong answers to my questions,
I'll definitely need guidance.

I do know that video-frequency [in B&W video, not color] has two
elements:

1. Temporal frequency

2. Spatial frequency

#1 only applies if the video consists of changing visual signals [such
as a movie or show]

#2 applies to all video signals -- including still images.

In color video, there is the a 3rd element [which is irrelevant to
this discussion] and that relates to the wavelengths of lights in the
video.

 

[Jerry Avins]

Radium wrote:

...

I want the actress to talk at the same speed, at a lower-pitch, and
finish at the same-time without any low-pass filtering.

That's audio pitch shifting. What has it to do with video? You wanted
something else in the past.

Jerry

 

[Radium]

Radium wrote:

That's audio pitch shifting. What has it to do with video?

I want the video-equivalent of that.