How to Convert Audio From 2D to 3D

Immersive audio is becoming an increasingly important factor for improving user experience in the fields of music, gaming, and audio/video editing. The spatial sound function is ideal for meetings and sports rehabilitation, especially for exhibitions, as it helps provide a more immersive experience. For users with visual impairments, the function can be a helpful guide.

In this article, I will reuse the sample code in this GitHub repository. I will implement spatial audio functions in my android app and render 3D surround sound.

preparations

Prepare the audio for 2D to 3D conversion, which is better as an MP3 file. If not, follow the instructions given later to convert the format to MP3 first. If the audio is part of a video file, simply extract the audio first by referring to the instructions later.

1. Configure the Maven repository address in the build.gradle file at the project level.

buildscript {
    repositories {
        google()
        jcenter()
        // Configure the Maven repository address for the HMS Core SDK.
        maven {url 'https://developer.huawei.com/repo/'}
    }
    dependencies {
        ...
        // Add the AppGallery Connect plugin configuration.
        classpath 'com.huawei.agconnect:agcp:1.4.2.300'
    }
}
allprojects {
    repositories {
        google()
        jcenter()
        // Configure the Maven repository address for the HMS Core SDK.
        maven {url 'https://developer.huawei.com/repo/'}
    }
}

Add the following configuration under the declaration in the header of the file:

apply plugin: 'com.huawei.agconnect'

2. Add the build dependency to the Audio Editor SDK in the build.gradle file at the application level.

dependencies{
    implementation 'com.huawei.hms:audio-editor-ui:{version}'
}

3. Apply for the following permissions in the AndroidManifest.xml file:

<!-- Vibrate -->
<uses-permission android:name="android.permission.VIBRATE" ></uses>
<!-- Microphone -->
<uses-permission android:name="android.permission.RECORD_AUDIO" ></uses>
<!-- Write into storage -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" ></uses>
<!-- Read from storage -->
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" ></uses>
<!-- Connect to the Internet -->
<uses-permission android:name="android.permission.INTERNET" ></uses>
<!-- Obtain the network status -->
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" ></uses>
<!-- Obtain the changed network connectivity state -->
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" ></uses>

Code development

1. Create the app’s custom activity to select one or more audio files. Return their paths to the SDK.

// Return the audio file paths to the audio editing screen.
private void sendAudioToSdk() {
    // Set filePath to the obtained audio file path.
    String filePath = "/sdcard/AudioEdit/audio/music.aac";
    ArrayList<String> audioList = new ArrayList<>();
    audioList.add(filePath);
    // Return the path to the audio editing screen.
    Intent intent = new Intent();
    // Use HAEConstant.AUDIO_PATH_LIST provided by the SDK.
    intent.putExtra(HAEConstant.AUDIO_PATH_LIST, audioList);
    // Use HAEConstant.RESULT_CODE provided by the SDK as the result code.
    this.setResult(HAEConstant.RESULT_CODE, intent);
    finish();
}

2. Record the activity in the AndroidManifest.xml file as shown in the following code. When you choose to import the selected audio files, the SDK will send a target whose action value is com.huawei.hms.audioeditor.chooseaudio to jump to the activity.

<activity android:name="Activity "> 
<intent-filter> 
<action android:name="com.huawei.hms.audioeditor.chooseaudio"></action> 
<category android:name="android.intent.category.DEFAULT"></category> 
</intent-filter> 
</activity>

Turn on the audio editing screen. When you click Add Audio, the SDK will automatically invoke the predefined activity. Then operations such as editing and adding special effects to the sound can be performed. After these operations are completed, the edited audio can be exported.

HAEUIManager.getInstance().launchEditorActivity(this);

3. Convert the file format to MP3. (my choice)

communicate transliteration To convert the format and save the converted audio to the default directory.

// Convert the audio format.
HAEAudioExpansion.getInstance().transformAudioUseDefaultPath(context,inAudioPath, audioFormat, new OnTransformCallBack() {
    // Callback when the progress is received. The value ranges from 0 to 100.
    @Override
    public void onProgress(int progress) {
    }
    // Callback when the conversion fails.
    @Override
    public void onFail(int errorCode) {
    }
    // Callback when the conversion succeeds.
    @Override
    public void onSuccess(String outPutPath) {
    }
    // Callback when the conversion is canceled.
    @Override
    public void onCancel() {
    }
    });
// Cancel format conversion.
HAEAudioExpansion.getInstance().cancelTransformAudio();

communicate audio converter To convert audio and save the converted audio to a specified directory.

// Convert the audio format.
HAEAudioExpansion.getInstance().transformAudio(context,inAudioPath, outAudioPath, new OnTransformCallBack(){
    // Callback when the progress is received. The value ranges from 0 to 100.
    @Override
    public void onProgress(int progress) {
    }
    // Callback when the conversion fails.
    @Override
    public void onFail(int errorCode) {
    }
    // Callback when the conversion succeeds.
    @Override
    public void onSuccess(String outPutPath) {
    }
    // Callback when the conversion is canceled.
    @Override
    public void onCancel() {
    }
    });
// Cancel format conversion.
HAEAudioExpansion.getInstance().cancelTransformAudio();

Call extractAudio Extract audio from a video to a specified directory. (my choice)

// outAudioDir (optional): directory path for storing extracted audio.
// outAudioName (optional): name of extracted audio, which does not contain the file name extension.
HAEAudioExpansion.getInstance().extractAudio(context,inVideoPath,outAudioDir, outAudioName,new AudioExtractCallBack() {
    @Override
    public void onSuccess(String audioPath) {
    Log.d(TAG, "ExtractAudio onSuccess : " + audioPath);
    }
    @Override
    public void onProgress(int progress) {
    Log.d(TAG, "ExtractAudio onProgress : " + progress);
    }
    @Override
    public void onFail(int errCode) {
    Log.i(TAG, "ExtractAudio onFail : " + errCode);
    }
    @Override
    public void onCancel() {
    Log.d(TAG, "ExtractAudio onCancel.");
    }
    });
// Cancel audio extraction.
HAEAudioExpansion.getInstance().cancelExtractAudio();

Call getInstruments and startSeparationTasks to separate the audio source.

// Obtain the accompaniment ID using getInstruments and pass the ID to startSeparationTasks.
HAEAudioSeparationFile haeAudioSeparationFile = new HAEAudioSeparationFile();
haeAudioSeparationFile.getInstruments(new SeparationCloudCallBack<List<SeparationBean>>() {
    @Override
public void onFinish(List<SeparationBean> response) {
// Callback when the separation data is received. The data includes the accompaniment ID.
}
    @Override
    public void onError(int errorCode) {
        // Callback when the separation fails.
}
});
// Set the parameter for accompaniment separation.
List instruments = new ArrayList<>();
instruments.add("accompaniment ID");
haeAudioSeparationFile.setInstruments(instruments);
// Start separating.
haeAudioSeparationFile.startSeparationTasks(inAudioPath, outAudioDir, outAudioName, new AudioSeparationCallBack() {
    @Override
    public void onResult(SeparationBean separationBean) { }
    @Override
    public void onFinish(List<SeparationBean> separationBeans) {}
    @Override
    public void onFail(int errorCode) {}
    @Override
    public void onCancel() {}
});
// Cancel separating.
haeAudioSeparationFile.cancel();

Call the applicationAudioFile for the spatial audio application.

// Apply spatial audio.
// Fixed position mode.
HAESpaceRenderFile haeSpaceRenderFile = new HAESpaceRenderFile(SpaceRenderMode.POSITION);
haeSpaceRenderFile.setSpacePositionParams(
                            new SpaceRenderPositionParams(x, y, z));
// Dynamic rendering mode.
HAESpaceRenderFile haeSpaceRenderFile = new HAESpaceRenderFile(SpaceRenderMode.ROTATION);
haeSpaceRenderFile.setRotationParams( new SpaceRenderRotationParams(
                                    x, y, z, surroundTime, surroundDirection));
// Extension.
HAESpaceRenderFile haeSpaceRenderFile = new HAESpaceRenderFile(SpaceRenderMode.EXTENSION);
haeSpaceRenderFile.setExtensionParams(new SpaceRenderExtensionParams(radiusVal, angledVal));
// Call the API.
haeSpaceRenderFile.applyAudioFile(inAudioPath, outAudioDir, outAudioName, callBack);
// Cancel applying spatial audio.
haeSpaceRenderFile.cancel();

After completing these steps, you can now perform a 2D to 3D conversion effect for your app.

.

Leave a Comment