Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Starting new audio lib #1248

Merged
merged 11 commits into from
Nov 23, 2015
Merged

Starting new audio lib #1248

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
48d8809
Starting new audio lib
alexjsmac Oct 31, 2015
97715c0
Merge remote-tracking branch 'origin/master' into new_sound_lib
alexjsmac Nov 1, 2015
05d0e16
Added sound effect block
alexjsmac Nov 2, 2015
dc69ef7
Made lists for audio wave options
alexjsmac Nov 6, 2015
c0863ef
Instrument, play, and rest blocks
alexjsmac Nov 6, 2015
519b3fe
Different notes now supported
alexjsmac Nov 12, 2015
a188a92
Added mml block and tempo block
alexjsmac Nov 14, 2015
d034417
New sound effect
alexjsmac Nov 14, 2015
057e2ce
Envelope options, playback audio
alexjsmac Nov 14, 2015
5729cda
Can write music with blocks now
alexjsmac Nov 21, 2015
2fef54e
Merge branch 'master' into new_sound_lib
alexjsmac Nov 23, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions css/block.css
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ wb-value[type*=text] > input{background-image: url(../images/icon/text.svg);}
wb-value[type*=boolean] > input{background-image: url(../images/icon/boolean.svg);}
wb-value[type*=sprite] > input{background-image: url(../images/icon/sprite.svg);}
wb-value[type*=sound] > input{background-image: url(../images/icon/sound.svg);}
wb-value[type*=audio] > input{ background-image: url(../images/icon/audio.svg);}
wb-value[type*=array] > input{background-image: url(../images/icon/array.svg);}
wb-value[type*=image] > input{background-image: url(../images/icon/image.svg);}
wb-value[type*=shape] > input{background-image: url(../images/icon/shape.svg);}
Expand Down Expand Up @@ -282,6 +283,7 @@ wb-expression[type=boolean]:before{ background-image: url(../images/icon/boolean
wb-expression[type=sprite]:before{ background-image: url(../images/icon/sprite.svg); }
wb-expression[type=any]:before{ background-image: url(../images/icon/control.svg); }
wb-expression[type=sound]:before{ background-image: url(../images/icon/sound.svg); }
wb-expression[type=audio]:before{ background-image: url(../images/icon/audio.svg); }
wb-expression[type=array]:before{ background-image: url(../images/icon/array.svg); }
wb-expression[type=wb-image]:before{ background-image: url(../images/icon/image.svg); }
wb-expression[type=shape]:before{ background-image: url(../images/icon/shape.svg); }
Expand Down
19 changes: 19 additions & 0 deletions images/icon/audio.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
198 changes: 157 additions & 41 deletions js/runtime.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
(function(global){
'use strict';

// Dependencies: ctx, canvas, Event, runtime, sound, soundEffect,
// Dependencies: ctx, canvas, Event, runtime
// canvas/stage stuff
var _canvas, _ctx;
var song = "o4 l4 V12 ";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is song needed here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah just need to set it up with those parameters specifying the octave, default volume, and volume before notes begin being added.

var current_octave = 4;
function canvas(){
if (!_canvas){
if (dom.find){
Expand Down Expand Up @@ -821,51 +823,165 @@
},

sound: {

get: function(url){
return assets.sounds[url]; // already cached by sounds library
},
play: function(sound){
sound.play();
},
setLoop: function(sound, flag){
sound.loop = flag;
},
setVolume: function(sound, volume){
sound.volume = volume;
get: function(wave, a, r){
var osc = T(wave);
var env = T("perc", {a:a, r:r});
var oscenv = T("OscGen", {osc:osc, env:env, mul:0.15}).play();
return oscenv;
},
getAudio: function(file){
var audio = T("audio", {load:file});
return audio;
},
playNote: function(note, octave, beats){
switch(note){
case "A":
note = "a";
break;
case "A#/Bb":
note = "a#";
break;
case "B":
note = "b";
break;
case "C":
note = "c";
break;
case "C#/Db":
note = "c#";
break;
case "D":
note = "d";
break;
case "D#/Eb":
note = "d#";
break;
case "E":
note = "e";
break;
case "F":
note = "f";
break;
case "F#/Gb":
note = "f#";
break;
case "G":
note = "g";
break;
case "G#/Ab":
note = "g#";
break;
case "Rest":
note = "r";
break;
}
if (octave > current_octave) {
var octave_diff = octave - current_octave;
for (var i = 0; i < octave_diff; i++) {
song += "<";
}
}
else if (octave < current_octave) {
var octave_diff = current_octave - octave;
for (var i = 0; i < octave_diff; i++) {
song += ">";
}
}
current_octave = octave;
var length;
switch(beats){
case "1/32":
length = "32";
break;
case "1/16":
length = "16";
break;
case "1/8":
length = "8";
break;
case "1/4":
length = "4";
break;
case "1/2":
length = "2";
break;
case "1":
length = "1";
break;
}
var newNote = note + length;
song += newNote;
},
playAudio: function(audio){
audio.play();
},
playInstrument: function(sound){
console.log(song);
T("mml", {mml:song}, sound).on("ended", function() {
sound.pause();
this.stop();
}).start();
song = "o4 l4 V12 ";
},
mml: function(sound, mml){
var gen = T("OscGen", {wave:sound, env:{type:"perc"}, mul:0.25}).play();
T("mml", {mml:mml}, gen).on("ended", function() {
gen.pause();
this.stop();
}).start();
},
tempo: function(tempo){
song += ("t" + tempo + " ");
},
pause: function(sound){
sound.pause();
},
playFrom: function(sound, time){
sound.playFrom(time);
},
pan: function(sound, balance){
sound.pan = balance;
},
echo_DelayFeedbackFilter: function(sound, delay, feedback, filter){
sound.setEcho(delay, feedback, filter);
},
stopEcho: function(sound){
sound.echo = false;
},
reverb_DurationDecayReverse: function(sound, duration, decay, reverse){
sound.setReverb(duration, decay, reverse);
},
stopReverb: function(sound){
sound.reverb = false;
keys: function(wave, vol){
var synth = T("OscGen", {wave:wave, mul:vol}).play();

var keydict = T("ndict.key");
var midicps = T("midicps");
T("keyboard").on("keydown", function(e) {
var midi = keydict.at(e.keyCode);
if (midi) {
var freq = midicps.at(midi);
synth.noteOnWithFreq(freq, 100);
}
}).on("keyup", function(e) {
var midi = keydict.at(e.keyCode);
if (midi) {
synth.noteOff(midi, 100);
}
}).start();

alert("Play notes on the keyboard");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this alert for debugging, or is it intended to be part of the interface?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it's intended to notify users that the keyboard is now playable. Though I suppose it would just be annoying if someone already knows that.

},
effect: function(frequency, attack, decay, wait, echoDelay, echoFeedback, echoFilter, waveform, volume, balance, pitchBend, reverseBend, random, dissonance){
return {
play: function(){
soundEffect(
frequency, attack, decay, waveform,
volume, balance, wait,
pitchBend, reverseBend, random, dissonance,
[echoDelay, echoFeedback, echoFilter]
);
}
};
effect: function(effect){
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way for users to add their own effects, or does that require modifying the blocks and runtime?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if eventually sound effects could be made from audio files using a resampling algorithm, sort of similiar to how HyperCard played sounds.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for stealing as many ideas from HyperCard as possible. Where can I learn more about the resampling algorithm it used?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I imagine it would be pretty difficult to learn about the specific algorithm that HyperCard used, as it involved proprietary APIs within the classic Macintosh Toolkit. I think resampling in general involves a combination of changing the sample rate (eg, if it's 44.1 kHz, treat it as 22 kHz instead) and a transformation to correct for the fact that simply changing the sample rate also changes the temporal length of the sound (so, either interpolating or dropping samples would be the simplistic approach; maybe there's something more sophisticated though). If the goal is to get a specific resultant pitch, you would also need some way to determine the native pitch of the sample.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, users can only choose from the sound effects listed but we can definitely add a block form them to define their own. I'm not familiar with HyperCard but I will check it out!

switch(effect) {
case "laser":
var table = [1760, [110, "200ms"]];

var freq = T("env", {table:table}).on("bang", function() {
VCO.mul = 0.2;
}).on("ended", function() {
VCO.mul = 0;
});
var VCO = T("saw", {freq:freq, mul:0}).play();
freq.bang();
break;
case "alarm":
var table = [440, [880, 500], [660, 250]];
var env = T("env", {table:table}).bang();
var synth = T("saw", {freq:env, mul:0.25});

var interval = T("interval", {interval:1000}, function(count) {
if (count === 3) {
interval.stop();
}
env.bang();
}).set({buddies:synth}).start();
break;
}
}
},

Expand Down
1 change: 1 addition & 0 deletions karma.conf.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ module.exports = function(config) {

// list of files / patterns to load in the browser
files: [
{ pattern: 'lib/timbre.js', watched: false},
{ pattern: 'lib/*.js', watched: false },
'js/util.js',
'js/event.js',
Expand Down
134 changes: 134 additions & 0 deletions lib/keyboard.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
(function(T) {
"use strict";

if (T.envtype !== "browser") {
return;
}

var fn = T.fn;
var instance = null;

function KeyboardListener(_args) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a new keyboard listener? We have a global one already, can that be extended for the purpose here, or is it important that this only listen for keys pressed while the playground is focused?

if (instance) {
return instance;
}
instance = this;

T.Object.call(this, 1, _args);

fn.fixKR(this);
}
fn.extend(KeyboardListener);

var keyDown = {};
var shiftKey = false;
var ctrlKey = false;
var altKey = false;

var onkeydown = function(e) {
var _ = instance._;
var cell = instance.cells[0];
var value = e.keyCode * _.mul + _.add;

for (var i = 0, imax = cell.length; i < imax; ++i) {
cell[i] = value;
}
shiftKey = e.shiftKey;
ctrlKey = e.ctrlKey;
altKey = e.altKey;

if (!keyDown[e.keyCode]) {
keyDown[e.keyCode] = true;
instance._.emit("keydown", e);
}
};

var onkeyup = function(e) {
delete keyDown[e.keyCode];
instance._.emit("keyup", e);
};

var $ = KeyboardListener.prototype;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using $ as a variable is going to confuse people who are used to using jQuery and associate $ with the jQuery object. Is there a reason not to use a more descriptive name like keyListenerProto?


Object.defineProperties($, {
shiftKey: {
get: function() {
return shiftKey;
}
},
ctrlKey: {
get: function() {
return ctrlKey;
}
},
altKey: {
get: function() {
return altKey;
}
}
});

$.start = function() {
window.addEventListener("keydown", onkeydown, true);
window.addEventListener("keyup" , onkeyup , true);
return this;
};

$.stop = function() {
window.removeEventListener("keydown", onkeydown, true);
window.removeEventListener("keyup" , onkeyup , true);
return this;
};

$.play = $.pause = function() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So play() and pause() are no-ops?

return this;
};

fn.register("keyboard", KeyboardListener);


var NDictKey = {
90 : 48, // Z -> C3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the comments here showing mapping of keys to notes. Is this exposed to the user as well in some way? How do they learn the mapping?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to us to decide that. Attached is a good image. But how to include it with the block?

keymap

83 : 49, // S -> C+3
88 : 50, // X -> D3
68 : 51, // D -> D+3
67 : 52, // C -> E3
86 : 53, // V -> F3
71 : 54, // G -> F+3
66 : 55, // B -> G3
72 : 56, // H -> G+3
78 : 57, // N -> A3
74 : 58, // J -> A+3
77 : 59, // M -> B3
188: 60, // , -> C4
76 : 61, // L -> C+4
190: 62, // . -> D4
186: 63, // ; -> D+4

81 : 60, // Q -> C4
50 : 61, // 2 -> C+4
87 : 62, // W -> D4
51 : 63, // 3 -> D+4
69 : 64, // E -> E4
82 : 65, // R -> F4
53 : 66, // 5 -> F+4
84 : 67, // T -> G4
54 : 68, // 6 -> G+4
89 : 69, // Y -> A4
55 : 70, // 7 -> A+4
85 : 71, // U -> B4
73 : 72, // I -> C5
57 : 73, // 9 -> C#5
79 : 74, // O -> D5
48 : 75, // 0 -> D+5
80 : 76 // P -> E5
};

var NDictNode = fn.getClass("ndict");
fn.register("ndict.key", function(_args) {
var instance = new NDictNode(_args);
instance.dict = NDictKey;
return instance;
});

})(timbre);
Loading