Thing Translator

• Download ThingTranslator2.zip - 5.2 MB

Introduction

In Part I of my tutorial we built a Web API that handles image labeling and translation into different languages. Now we will build Android app that can consume that API. Features we are going to have are:

• select destination language
• select image from gallery or take new photo using camera
• option to toogle speech output on/off

Using the code

All the code base is located in code section of this article. For simplicity reasons here I am going to highlight only the important parts of the code.

build.gradle

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    compile 'com.google.code.gson:gson:2.6.1'
    compile 'com.android.support:support-v13:23.4.0'
    compile 'com.android.support:appcompat-v7:23.4.0'
    compile 'com.android.support:design:23.4.0'
    compile 'com.squareup.retrofit2:retrofit:2.0.0'
    compile 'com.squareup.retrofit2:converter-gson:2.0.0'
    compile 'com.squareup.okhttp3:okhttp:3.4.1'
    compile 'com.android.support.constraint:constraint-layout:1.0.0-alpha4'
    testCompile 'junit:junit:4.12'
    compile 'com.pixplicity.easyprefs:library:1.8.1@aar'
    compile 'com.jakewharton:butterknife:8.4.0'
    annotationProcessor 'com.jakewharton:butterknife-compiler:8.4.0'
}

We will use some really cool Java libraries that makes Android development lot easier.

• easyprefs -  easy access to SharedPrefferences
• butterknife - makes Views declaration lot easier
• retrofit - enables us to work with REST API's quickly and easily 

activity_main - spinner View has a custom layout defined in spinner_item.xml

<Spinner
    android:id="@+id/spinner"
    android:layout_width="100dp"
    android:layout_height="match_parent"
    android:layout_marginBottom="16dp"
    android:layout_weight="0.91"
    android:gravity="bottom|right" />

<?xml version="1.0" encoding="utf-8"?>
<TextView
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="wrap_content"
    android:layout_height="30dp"
    android:textColor="@android:color/darker_gray"
    android:textSize="24dp"
    />

ApiInterface.java - this is interface for our REST API calls. Put your API url we build in previous tutorial into ENDPOINT variable. As you can see only 2 parameters are used when making POST request:

1. Image file
2. Language Code

package thingtranslator2.jalle.com.thingtranslator2.Rest;
 

public interface ApiInterface {
    
    String ENDPOINT = "YOUR-API-URL/api/";

    @Multipart
    @POST("upload")
    Call<translation> upload(@Part("image\"; filename=\"pic.jpg\" ") RequestBody file, @Part("FirstName") RequestBody langCode1);

    final OkHttpClient okHttpClient = new OkHttpClient.Builder()
            .readTimeout(60, TimeUnit.SECONDS)
            .connectTimeout(60, TimeUnit.SECONDS)
            .build();

    Retrofit retrofit = new Retrofit.Builder()
            .baseUrl(ENDPOINT)
            .client(okHttpClient)
            .addConverterFactory(GsonConverterFactory.create())
            .build();
}

Translation class

package thingtranslator2.jalle.com.thingtranslator2.Rest;

public class Translation {
    public String Original;
    public String Translation;
    public String Error;
}

Tools - ImagePicker this class is used for getting image from gallery of taking new photo from camera. Tools - MarshMallowPermission this is used for setting up camera permissions on Android 6.

AndroidManifest

    <?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="thingtranslator2.jalle.com.thingtranslator2">

    <uses-feature android:name="android.hardware.camera" />
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-feature android:name="android.hardware.camera.autofocus" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity" >

            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>
</manifest>

MainActivity.java

public class MainActivity extends AppCompatActivity implements AdapterView.OnItemSelectedListener {
    public
    @BindView(R.id.txtTranslation)
    TextView txtTranslation;
    public
    @BindView(R.id.spinner)
    Spinner spinner;
    public
    @BindView(R.id.imgPhoto)
    ImageButton imgPhoto;
    public
    @BindView(R.id.imgSpeaker)
    ImageButton btnSpeaker;

    public Boolean speakerOn, firstRun;
    public TextToSpeech tts;
    public List<string> languages = new ArrayList<string>();
    public ArrayList<language> languageList = new ArrayList<language>();
    public ArrayAdapter<language> spinnerArrayAdapter;
    public Language selectedLanguage;
    public static final int PICK_IMAGE_ID = 234; // the number doesn't matter
    thingtranslator2.jalle.com.thingtranslator2.MarshMallowPermission marshMallowPermission = new thingtranslator2.jalle.com.thingtranslator2.MarshMallowPermission(this);

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN);
        ButterKnife.bind(this);

        //init our Shared preferences helper
        new Prefs.Builder()
                .setContext(this)
                .setMode(ContextWrapper.MODE_PRIVATE)
                .setPrefsName(getPackageName())
                .setUseDefaultSharedPreference(true)
                .build();

        //  Prefs.clear();
        firstRun = Prefs.getBoolean("firstRun", true);
        spinner.setOnItemSelectedListener(this);
        tts = new TextToSpeech(getApplicationContext(), new TextToSpeech.OnInitListener() {
            @Override
            public void onInit(int status) {
                if (status != TextToSpeech.ERROR) {
                    fillSpinner();


                }
            }
        });


        if (firstRun) {
            Prefs.putBoolean("firstRun", false);
            Prefs.putBoolean("speakerOn", true);
            Prefs.putString("langCode", "bs");
        }
        setSpeaker();
    }

    private void setSpeaker() {
        int id = Prefs.getBoolean("speakerOn", false) ? R.drawable.ic_volume : R.drawable.ic_volume_off;
        btnSpeaker.setImageBitmap(BitmapFactory.decodeResource(getResources(), id));
    }

    public void ToogleSpeaker(View v) {
        Prefs.putBoolean("speakerOn", !Prefs.getBoolean("speakerOn", false));
        setSpeaker();
        Toast.makeText(getApplicationContext(), "Speech " + (Prefs.getBoolean("speakerOn", true) ? "On" : "Off"), Toast.LENGTH_SHORT);
    }

    private void fillSpinner() {
        Iterator itr = tts.getAvailableLanguages().iterator();
        while (itr.hasNext()) {
            Locale item = (Locale) itr.next();
            languageList.add(new Language(item.getDisplayName(), item.getLanguage()));
        }
        //Sort that array
        Collections.sort(languageList, new Comparator<language>() {
            @Override
            public int compare(Language o1, Language o2) {
                return o1.getlangCode().compareTo(o2.getlangCode());
            }
        });

        spinnerArrayAdapter = new ArrayAdapter<language>(this, R.layout.spinner_item, languageList);
        spinnerArrayAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
        spinner.setAdapter(spinnerArrayAdapter);

        //Set the default language to bs- Bosnian
        String def = Prefs.getString("langCode", "bs");
        for (Language lang : languageList) {
            if (lang.getlangCode().equals(def)) {
                spinner.setSelection(languageList.indexOf(lang));
            }
        }
    }

    ;

    //Get image from  Gallery or Camera
    public void getImage(View v) {
        if (!marshMallowPermission.checkPermissionForCamera()) {
            marshMallowPermission.requestPermissionForCamera();
        } else {
            if (!marshMallowPermission.checkPermissionForExternalStorage()) {
                marshMallowPermission.requestPermissionForExternalStorage();
            } else {
                Intent chooseImageIntent = thingtranslator2.jalle.com.thingtranslator2.ImagePicker.getPickImageIntent(getApplicationContext());
                startActivityForResult(chooseImageIntent, PICK_IMAGE_ID);
            }
        }
    }


    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (data == null) return;

        switch (requestCode) {
            case PICK_IMAGE_ID:
                Bitmap bitmap = thingtranslator2.jalle.com.thingtranslator2.ImagePicker.getImageFromResult(this, resultCode, data);
                imgPhoto.setImageBitmap(null);
                imgPhoto.setBackground(null);
                imgPhoto.setImageBitmap(bitmap);
                imgPhoto.invalidate();
                imgPhoto.postInvalidate();

                File file = null;
                try {
                    file = savebitmap(bitmap, "pic.jpeg");
                } catch (IOException e) {
                    e.printStackTrace();
                }
                uploadImage(file);
                break;
            default:
                super.onActivityResult(requestCode, resultCode, data);
                break;
        }
    }

    private void uploadImage(File file) {
        RequestBody body = RequestBody.create(MediaType.parse("image/*"), file);
        RequestBody langCode1 = RequestBody.create(MediaType.parse("text/plain"), selectedLanguage.langCode);

        final ProgressDialog progress = new ProgressDialog(this);
        progress.setMessage("Processing image...");
        progress.setProgressStyle(ProgressDialog.STYLE_SPINNER);
        progress.setIndeterminate(true);
        progress.show();


        ApiInterface mApiService = ApiInterface.retrofit.create(ApiInterface.class);
        Call<translation> mService = mApiService.upload(body, langCode1);
        mService.enqueue(new Callback<translation>() {
            @Override
            public void onResponse(Call<translation> call, Response<translation> response) {
                progress.hide();
                Translation result = response.body();
                txtTranslation.setText(result.Translation);
                txtTranslation.invalidate();

                if (Prefs.getBoolean("speakerOn", true)) {
                    tts.speak(result.Translation, TextToSpeech.QUEUE_FLUSH, null);
                }
            }

            @Override
            public void onFailure(Call<translation> call, Throwable t) {
                call.cancel();
                progress.hide();
                Toast.makeText(getApplicationContext(), "Error: " + t.getMessage(), Toast.LENGTH_LONG).show();
            }
        });
    }


    public static File savebitmap(Bitmap bmp, String fName) throws IOException {
        ByteArrayOutputStream bytes = new ByteArrayOutputStream();
        bmp.compress(Bitmap.CompressFormat.JPEG, 60, bytes);
        File f = new File(Environment.getExternalStorageDirectory()
                + File.separator + fName);
        f.createNewFile();
        FileOutputStream fo = new FileOutputStream(f);
        fo.write(bytes.toByteArray());
        fo.close();
        return f;
    }


    @Override
    public void onItemSelected(AdapterView parent, View view, int position, long id) {

        selectedLanguage = (Language) spinner.getSelectedItem();
        //  langCode = languages.get(position);
        tts.setLanguage(Locale.forLanguageTag(selectedLanguage.langCode));
        Prefs.putString("langCode", selectedLanguage.langCode);
    }

    @Override
    public void onNothingSelected(AdapterView parent) {

    }
}

The end result of this tutorial is application that looks like this. Remember your WebAPI service needs to be up and running in order for all this to work.

Points of Interest

There are many ways for  improvement and upgrading this application. One of my ideas is that the image labeling could be done during the live preview on your camera. Having the preview window opened, we can invoke the API every 5-10 seconds and have live labeling about what we see through our camera.
One drawback is that  it would consume a lot of bandwidth and aslo remember, Google API Vision and most of the other APIs are not free. You pay for the API usage when requests exceed the allowed quota.

Happy codding!