php - Best strategy for doing real time, scalable audio processing? -

-

i building web application allows users upload audio files, music in particular. of time, expect duration of each song several minutes , file approximately 3-10mb in size. however, accept audio uploads 100mb, possibly allowing on hour of audio. using combination of ffmpeg, sox, , lame convert 7 possible formats mp3 , perform audio modifications including equalization, trimming, , fading. files stored , linked in database.

my current strategy handle entire process in 1 http file upload request using php on backend, in perform following functions:

  1. validation
  2. transcode audio multiple versions (using shell through php)
  3. store original , transcoded versions in temp directory
  4. upload audio files amazon s3 permanent storage
  5. commit id of each file database, linking them user

this works similar image processing system have set up. however, while images can complete whole process in few seconds, audio can take lot longer. @ most, audio take 5-10 minutes processed , stored.

my questions are:

  1. for audio processing, better fork off transcoding background process, writing state database, , pinging every few seconds update webpage vs. doing in 1 http request?

  2. with intention of scaling in future, advisable processing on single server instance, leaving frontend web instances free replicate / destroyed?

    • if yes, require cross-domain file uploading directly server? (anyone know if how youtube or big sites it?)

thanks!

if understand system correctly, best approach more this:

  • in web front-end, store audio , create "task" indicating audio needs processed.
  • run background task pulls tasks , processing. @ end of task, user can notified (if necessary) , database state can updated or whatever.

your tasks should written if fail partway through, can re-executed start without causing problems. can run multiple background tasks , web front-ends in architecture.

a way write tasks using message passing system amqp. there cheap services rabbitmq you. can, of course, build own on top of database, may require polling.

finally, might find faster , more efficient use service zencoder transcoding, because can parallelize work , handle more input formats, may not compatible processing.


Comments

Post a Comment

Popular posts from this blog

java - Run a .jar on Heroku -

-
i trying run .jar file on heroku, , can't find instructions anywhere on how this. have read wants me compile uploading heroku. possible, , if is, how this? if helps, trying run minecraft craftbukkit server, , don't have pom.xml file yet. you didn't provide details i'll try give , people reaches question way it. first of all, create heroku account , create heroku application (assumed have done that). second, need generate application jar , upload heroku. so, open terminal , install heroku cli (assuming unix terminal): $ sudo npm install -g heroku-cli you need have npm , nodejs installed, can check link: https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-16-04 once done that, can use npm update node version acceptable heroku-cli: $ sudo npm install -g n $ sudo n 7.10.0 and run: $ sudo npm install -g heroku-cli in order deploy jars, need install deploy plugin: $ sudo heroku plugins:install heroku-cli-deploy ...
Read more

java - Jtable duplicate Rows -

-
Image
when add new rows in jtable duplicate rows many times this method update jtable public final void tableaucomande(){ ((defaulttablemodel)tabcommande.getmodel()).getdatavector().clear(); session s=hibernateutil.getsessionfactory().opensession(); criteria cr1= s.createcriteria(commande.class); list <commande> lcs=cr1.list() ; criteria cr2= s.createcriteria(lignecommande.class); list <lignecommande> lck=cr2.list() ; if(lcs.size()!=0 && lck.size()!=0) { (commande lc:lcs) { for(lignecommande lig:lck){ ((defaulttablemodel)tabcommande.getmodel()).addrow(new object[]{ lc.getid_cmd(),lc.getnumcmd(),lc.getdate_cmd(),lc.getclient().getnomprenom(), lig.getproduit().getlib_prd(),lig.getquantite(),lig.getproduit().getid_prd() }); } } // tabcommande.setrowselectioninterval(tabcommande.getrowcount()-1,tabcommande.getrowcount()-1); } } this code add add data rows displaye...
Read more

validation - How to pass paramaters like unix into windows batch file -

-
i need pass various parameters .cmd file unix format file.cmd -configuration=value -source=value -flag but, try this: startlocal @echo off cls setlocal set cmdline=%* set configuration= set source= set badargs= set validation= goto main :splitargs echo splitargs(%*) if "%*" neq "" ( /f "tokens=1,2,* delims== " %%i in ("%*") call :assignkeyvalue %%i %%j & call :splitargs %%k ) goto :eof :assignkeyvalue echo assignkeyvalue(%1, %2) if /i %1==-configuration ( set configuration=%2 ) else if /i %1==-source ( set source=%2 ) else ( rem append unrecognised [key,value] badargs echo unknown key %1 set badargs=%badargs%[%1, %2] ) goto :eof :validate echo validating set validation=fail if defined configuration ( echo -configuration ok if defined source ( echo -source ok if not defined badargs ( set validation=success ) ) ) goto :eof :main cl...
Read more