当前位置：首页 > news >正文

YOLOv11安卓目标检测App完整开发指南

news 2025/10/10 11:38:29

YOLOv11安卓目标检测App完整开发指南

1. 项目概述

1.1 项目目标

开发一款基于YOLOv11的安卓移动端目标检测应用，实现：

实时相机拍照/视频流检测
离线模型推理（无需联网）
自动应用和模型更新
检测结果上传至MES系统
高性能、低延迟的用户体验

1.2 技术栈

编程语言: Kotlin/Java
深度学习框架: TensorFlow Lite / ONNX Runtime / NCNN
UI框架: Jetpack Compose / XML布局
网络库: Retrofit2 + OkHttp3
图像处理: CameraX + OpenCV
依赖注入: Hilt/Dagger2
数据库: Room Database

1.3 核心功能模块

┌─────────────────────────────────────┐
│         安卓App主应用               │
├─────────────────────────────────────┤
│  相机模块  │  检测模块  │  更新模块  │
│  拍照/录像 │  YOLOv11  │  APK/模型  │
├─────────────────────────────────────┤
│  结果处理  │  数据存储  │  网络通信  │
│  可视化   │   本地DB   │  MES对接   │
└─────────────────────────────────────┘

2. 系统架构设计

2.1 整体架构

采用MVVM（Model-View-ViewModel）架构模式：

┌──────────────┐
│   View层     │  Activity/Fragment + Jetpack Compose
├──────────────┤
│ ViewModel层  │  业务逻辑 + LiveData/StateFlow
├──────────────┤
│  Repository  │  数据仓库层
├──────────────┤
│ Data Source  │  本地DB + 网络API + 模型推理
└──────────────┘

2.2 模块划分

app/
├── data/                    # 数据层
│   ├── local/              # 本地数据源
│   │   ├── dao/           # Room DAO
│   │   ├── database/      # 数据库
│   │   └── entity/        # 数据实体
│   ├── remote/            # 远程数据源
│   │   ├── api/          # API接口
│   │   └── dto/          # 数据传输对象
│   └── repository/        # 数据仓库
├── domain/                 # 业务逻辑层
│   ├── model/             # 业务模型
│   └── usecase/           # 用例
├── presentation/           # 展示层
│   ├── camera/            # 相机界面
│   ├── detection/         # 检测界面
│   └── settings/          # 设置界面
├── ml/                     # 机器学习模块
│   ├── detector/          # 检测器
│   ├── preprocessor/      # 预处理
│   └── postprocessor/     # 后处理
└── utils/                  # 工具类├── camera/            # 相机工具├── network/           # 网络工具└── update/            # 更新工具

3. 开发环境配置

3.1 Android Studio配置

// build.gradle (Project level)
buildscript {ext.kotlin_version = '1.9.20'ext.compose_version = '1.5.4'repositories {google()mavenCentral()}dependencies {classpath 'com.android.tools.build:gradle:8.2.0'classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"classpath 'com.google.dagger:hilt-android-gradle-plugin:2.48'}
}

3.2 App模块依赖

// build.gradle (App level)
plugins {id 'com.android.application'id 'kotlin-android'id 'kotlin-kapt'id 'dagger.hilt.android.plugin'
}android {namespace 'com.example.yolodetector'compileSdk 34defaultConfig {applicationId "com.example.yolodetector"minSdk 24targetSdk 34versionCode 1versionName "1.0.0"// 支持armeabi-v7a和arm64-v8andk {abiFilters 'armeabi-v7a', 'arm64-v8a'}}buildFeatures {viewBinding truecompose truemlModelBinding true}composeOptions {kotlinCompilerExtensionVersion = compose_version}packagingOptions {pickFirst 'lib/arm64-v8a/libc++_shared.so'pickFirst 'lib/armeabi-v7a/libc++_shared.so'}
}dependencies {// Kotlinimplementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"implementation 'androidx.core:core-ktx:1.12.0'// UIimplementation 'androidx.appcompat:appcompat:1.6.1'implementation 'com.google.android.material:material:1.11.0'implementation "androidx.compose.ui:ui:$compose_version"implementation "androidx.compose.material3:material3:1.1.2"// CameraXdef camerax_version = "1.3.1"implementation "androidx.camera:camera-core:$camerax_version"implementation "androidx.camera:camera-camera2:$camerax_version"implementation "androidx.camera:camera-lifecycle:$camerax_version"implementation "androidx.camera:camera-view:$camerax_version"// TensorFlow Liteimplementation 'org.tensorflow:tensorflow-lite:2.14.0'implementation 'org.tensorflow:tensorflow-lite-gpu:2.14.0'implementation 'org.tensorflow:tensorflow-lite-support:0.4.4'// ONNX Runtime (可选)implementation 'com.microsoft.onnxruntime:onnxruntime-android:1.16.3'// OpenCVimplementation 'org.opencv:opencv:4.8.0'// 网络请求implementation 'com.squareup.retrofit2:retrofit:2.9.0'implementation 'com.squareup.retrofit2:converter-gson:2.9.0'implementation 'com.squareup.okhttp3:okhttp:4.12.0'implementation 'com.squareup.okhttp3:logging-interceptor:4.12.0'// 依赖注入implementation 'com.google.dagger:hilt-android:2.48'kapt 'com.google.dagger:hilt-compiler:2.48'// 数据库def room_version = "2.6.1"implementation "androidx.room:room-runtime:$room_version"implementation "androidx.room:room-ktx:$room_version"kapt "androidx.room:room-compiler:$room_version"// 协程implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3'// ViewModelimplementation 'androidx.lifecycle:lifecycle-viewmodel-ktx:2.7.0'implementation 'androidx.lifecycle:lifecycle-livedata-ktx:2.7.0'
}

3.3 权限配置

<!-- AndroidManifest.xml -->
<manifest xmlns:android="http://schemas.android.com/apk/res/android"><!-- 相机权限 --><uses-feature android:name="android.hardware.camera" /><uses-feature android:name="android.hardware.camera.autofocus" /><uses-permission android:name="android.permission.CAMERA" /><!-- 存储权限 --><uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /><uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /><!-- 网络权限 --><uses-permission android:name="android.permission.INTERNET" /><uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" /><!-- 更新安装权限 --><uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES" /><uses-permission android:name="android.permission.DOWNLOAD_WITHOUT_NOTIFICATION" /><applicationandroid:name=".YoloApplication"android:allowBackup="true"android:icon="@mipmap/ic_launcher"android:label="@string/app_name"android:usesCleartextTraffic="true"android:requestLegacyExternalStorage="true"><!-- FileProvider配置 --><providerandroid:name="androidx.core.content.FileProvider"android:authorities="${applicationId}.fileprovider"android:exported="false"android:grantUriPermissions="true"><meta-dataandroid:name="android.support.FILE_PROVIDER_PATHS"android:resource="@xml/file_paths" /></provider></application>
</manifest>

4. YOLOv11模型准备与转换

4.1 模型导出

使用Python将YOLOv11模型转换为移动端格式：

from ultralytics import YOLO# 加载训练好的模型
model = YOLO('yolov11n.pt')  # 或使用自己训练的模型# 方案1: 转换为TensorFlow Lite
model.export(format='tflite', imgsz=640, int8=False)# 方案2: 转换为ONNX
model.export(format='onnx', imgsz=640, simplify=True, dynamic=False)# 方案3: 转换为NCNN (适合移动端)
model.export(format='ncnn', imgsz=640)

4.2 模型量化（可选，减小体积）

import tensorflow as tf# INT8量化
converter = tf.lite.TFLiteConverter.from_saved_model('yolov11_saved_model')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.int8]# 提供代表性数据集
def representative_dataset():for _ in range(100):yield [np.random.rand(1, 640, 640, 3).astype(np.float32)]converter.representative_dataset = representative_dataset
tflite_quant_model = converter.convert()with open('yolov11_int8.tflite', 'wb') as f:f.write(tflite_quant_model)

4.3 模型文件放置

将转换后的模型文件放入Android项目：

app/src/main/assets/
├── yolov11.tflite          # TFLite模型
├── labels.txt              # 类别标签
└── config.json             # 模型配置

labels.txt示例：

person
bicycle
car
motorcycle
...

config.json示例：

{"model_name": "yolov11n","input_size": 640,"num_classes": 80,"conf_threshold": 0.25,"iou_threshold": 0.45,"max_detections": 300,"normalization": {"mean": [0, 0, 0],"std": [255, 255, 255]}
}

5. 安卓App核心功能实现

5.1 Application类初始化

@HiltAndroidApp
class YoloApplication : Application() {override fun onCreate() {super.onCreate()// 初始化OpenCVif (!OpenCVLoader.initDebug()) {Log.e("OpenCV", "Unable to load OpenCV!")}// 初始化全局配置AppConfig.init(this)}
}

5.2 依赖注入配置

@Module
@InstallIn(SingletonComponent::class)
object AppModule {@Provides@Singletonfun provideContext(application: Application): Context {return application.applicationContext}@Provides@Singletonfun provideYoloDetector(context: Context): YoloDetector {return YoloDetector(context)}@Provides@Singletonfun provideRetrofit(): Retrofit {return Retrofit.Builder().baseUrl(BuildConfig.BASE_URL).addConverterFactory(GsonConverterFactory.create()).client(OkHttpClient.Builder().addInterceptor(HttpLoggingInterceptor().apply {level = HttpLoggingInterceptor.Level.BODY}).connectTimeout(30, TimeUnit.SECONDS).readTimeout(30, TimeUnit.SECONDS).build()).build()}@Provides@Singletonfun provideMesApi(retrofit: Retrofit): MesApi {return retrofit.create(MesApi::class.java)}
}

5.3 数据库设计

// 检测结果实体
@Entity(tableName = "detection_results")
data class DetectionResult(@PrimaryKey(autoGenerate = true)val id: Long = 0,val timestamp: Long,val imagePath: String,val detections: String,  // JSON格式的检测框信息val isSynced: Boolean = false,val mesOrderId: String? = null
)// DAO接口
@Dao
interface DetectionDao {@Insertsuspend fun insert(result: DetectionResult): Long@Query("SELECT * FROM detection_results WHERE isSynced = 0 ORDER BY timestamp DESC")fun getUnsyncedResults(): Flow<List<DetectionResult>>@Query("UPDATE detection_results SET isSynced = 1 WHERE id = :id")suspend fun markAsSynced(id: Long)@Query("SELECT * FROM detection_results ORDER BY timestamp DESC LIMIT :limit")fun getRecentResults(limit: Int): Flow<List<DetectionResult>>
}// 数据库
@Database(entities = [DetectionResult::class], version = 1)
abstract class AppDatabase : RoomDatabase() {abstract fun detectionDao(): DetectionDao
}

6. 自动更新机制

6.1 更新检查服务

data class UpdateInfo(val versionCode: Int,val versionName: String,val apkUrl: String,val modelUrl: String?,val updateMessage: String,val forceUpdate: Boolean,val md5: String
)interface UpdateApi {@GET("api/version/check")suspend fun checkUpdate(@Query("currentVersion") currentVersion: Int,@Query("packageName") packageName: String): Response<UpdateInfo>@Streaming@GETsuspend fun downloadFile(@Url fileUrl: String): Response<ResponseBody>
}

6.2 更新管理器

class UpdateManager @Inject constructor(private val context: Context,private val updateApi: UpdateApi
) {private val downloadDir = File(context.getExternalFilesDir(null), "downloads")init {if (!downloadDir.exists()) {downloadDir.mkdirs()}}// 检查更新suspend fun checkForUpdates(): UpdateInfo? {return try {val currentVersion = context.packageManager.getPackageInfo(context.packageName, 0).versionCodeval response = updateApi.checkUpdate(currentVersion,context.packageName)if (response.isSuccessful && response.body() != null) {val updateInfo = response.body()!!if (updateInfo.versionCode > currentVersion) {updateInfo} else {null}} else {null}} catch (e: Exception) {Log.e("UpdateManager", "检查更新失败", e)null}}// 下载APKsuspend fun downloadApk(url: String,onProgress: (Int) -> Unit): File? = withContext(Dispatchers.IO) {try {val response = updateApi.downloadFile(url)if (!response.isSuccessful) return@withContext nullval apkFile = File(downloadDir, "update_${System.currentTimeMillis()}.apk")val body = response.body() ?: return@withContext nullval totalBytes = body.contentLength()val inputStream = body.byteStream()val outputStream = FileOutputStream(apkFile)val buffer = ByteArray(8192)var downloadedBytes = 0Lvar bytesRead: Intwhile (inputStream.read(buffer).also { bytesRead = it } != -1) {outputStream.write(buffer, 0, bytesRead)downloadedBytes += bytesReadval progress = ((downloadedBytes * 100) / totalBytes).toInt()withContext(Dispatchers.Main) {onProgress(progress)}}outputStream.close()inputStream.close()apkFile} catch (e: Exception) {Log.e("UpdateManager", "下载APK失败", e)null}}// 下载模型文件suspend fun downloadModel(url: String,onProgress: (Int) -> Unit): File? = withContext(Dispatchers.IO) {try {val response = updateApi.downloadFile(url)if (!response.isSuccessful) return@withContext nullval modelFile = File(context.filesDir, "models/yolov11_new.tflite")modelFile.parentFile?.mkdirs()val body = response.body() ?: return@withContext nullval totalBytes = body.contentLength()val inputStream = body.byteStream()val outputStream = FileOutputStream(modelFile)val buffer = ByteArray(8192)var downloadedBytes = 0Lvar bytesRead: Intwhile (inputStream.read(buffer).also { bytesRead = it } != -1) {outputStream.write(buffer, 0, bytesRead)downloadedBytes += bytesReadval progress = ((downloadedBytes * 100) / totalBytes).toInt()withContext(Dispatchers.Main) {onProgress(progress)}}outputStream.close()inputStream.close()modelFile} catch (e: Exception) {Log.e("UpdateManager", "下载模型失败", e)null}}// 安装APKfun installApk(apkFile: File) {val intent = Intent(Intent.ACTION_VIEW)intent.flags = Intent.FLAG_ACTIVITY_NEW_TASKif (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {val uri = FileProvider.getUriForFile(context,"${context.packageName}.fileprovider",apkFile)intent.setDataAndType(uri, "application/vnd.android.package-archive")intent.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION)} else {intent.setDataAndType(Uri.fromFile(apkFile),"application/vnd.android.package-archive")}context.startActivity(intent)}// 验证MD5fun verifyMd5(file: File, expectedMd5: String): Boolean {return try {val md = MessageDigest.getInstance("MD5")val fis = FileInputStream(file)val buffer = ByteArray(8192)var bytesRead: Intwhile (fis.read(buffer).also { bytesRead = it } != -1) {md.update(buffer, 0, bytesRead)}fis.close()val digest = md.digest()val calculatedMd5 = digest.joinToString("") { "%02x".format(it) }calculatedMd5.equals(expectedMd5, ignoreCase = true)} catch (e: Exception) {false}}
}

6.3 更新ViewModel

@HiltViewModel
class UpdateViewModel @Inject constructor(private val updateManager: UpdateManager
) : ViewModel() {private val _updateState = MutableStateFlow<UpdateState>(UpdateState.Idle)val updateState: StateFlow<UpdateState> = _updateState.asStateFlow()private val _downloadProgress = MutableStateFlow(0)val downloadProgress: StateFlow<Int> = _downloadProgress.asStateFlow()fun checkForUpdates() {viewModelScope.launch {_updateState.value = UpdateState.Checkingval updateInfo = updateManager.checkForUpdates()_updateState.value = if (updateInfo != null) {UpdateState.Available(updateInfo)} else {UpdateState.NoUpdate}}}fun downloadAndInstall(updateInfo: UpdateInfo) {viewModelScope.launch {_updateState.value = UpdateState.Downloadingval apkFile = updateManager.downloadApk(updateInfo.apkUrl) { progress ->_downloadProgress.value = progress}if (apkFile != null) {if (updateManager.verifyMd5(apkFile, updateInfo.md5)) {_updateState.value = UpdateState.Downloaded(apkFile)updateManager.installApk(apkFile)} else {_updateState.value = UpdateState.Error("文件校验失败")}} else {_updateState.value = UpdateState.Error("下载失败")}}}
}sealed class UpdateState {object Idle : UpdateState()object Checking : UpdateState()object NoUpdate : UpdateState()data class Available(val updateInfo: UpdateInfo) : UpdateState()object Downloading : UpdateState()data class Downloaded(val file: File) : UpdateState()data class Error(val message: String) : UpdateState()
}

7. 相机拍照功能

7.1 CameraX实现

class CameraManager(private val context: Context) {private var imageCapture: ImageCapture? = nullprivate var imageAnalyzer: ImageAnalysis? = nullprivate var camera: Camera? = nullprivate var cameraProvider: ProcessCameraProvider? = null// 初始化相机fun startCamera(lifecycleOwner: LifecycleOwner,previewView: PreviewView,analyzer: ImageAnalysis.Analyzer) {val cameraProviderFuture = ProcessCameraProvider.getInstance(context)cameraProviderFuture.addListener({cameraProvider = cameraProviderFuture.get()// 预览val preview = Preview.Builder().build().also {it.setSurfaceProvider(previewView.surfaceProvider)}// 拍照imageCapture = ImageCapture.Builder().setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY).setTargetRotation(previewView.display.rotation).build()// 图像分析imageAnalyzer = ImageAnalysis.Builder().setTargetResolution(Size(640, 640)).setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST).setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888).build().also {it.setAnalyzer(ContextCompat.getMainExecutor(context), analyzer)}// 选择后置摄像头val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERAtry {// 解绑所有用例cameraProvider?.unbindAll()// 绑定用例camera = cameraProvider?.bindToLifecycle(lifecycleOwner,cameraSelector,preview,imageCapture,imageAnalyzer)} catch (e: Exception) {Log.e("CameraManager", "用例绑定失败", e)}}, ContextCompat.getMainExecutor(context))}// 拍照fun takePhoto(onImageCaptured: (File) -> Unit, onError: (Exception) -> Unit) {val imageCapture = imageCapture ?: returnval photoFile = File(context.getExternalFilesDir(Environment.DIRECTORY_PICTURES),"IMG_${System.currentTimeMillis()}.jpg")val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile).build()imageCapture.takePicture(outputOptions,ContextCompat.getMainExecutor(context),object : ImageCapture.OnImageSavedCallback {override fun onImageSaved(output: ImageCapture.OutputFileResults) {onImageCaptured(photoFile)}override fun onError(exception: ImageCaptureException) {onError(exception)}})}// 停止相机fun stopCamera() {cameraProvider?.unbindAll()}// 切换闪光灯fun toggleFlash() {camera?.cameraControl?.enableTorch(camera?.cameraInfo?.torchState?.value == TorchState.OFF)}
}

7.2 相机Activity

@AndroidEntryPoint
class CameraActivity : AppCompatActivity() {private lateinit var binding: ActivityCameraBindingprivate lateinit var cameraManager: CameraManager@Injectlateinit var yoloDetector: YoloDetectorprivate var isRealTimeDetection = falseoverride fun onCreate(savedInstanceState: Bundle?) {super.onCreate(savedInstanceState)binding = ActivityCameraBinding.inflate(layoutInflater)setContentView(binding.root)cameraManager = CameraManager(this)// 请求权限if (allPermissionsGranted()) {startCamera()} else {requestPermissions()}setupUI()}private fun startCamera() {cameraManager.startCamera(lifecycleOwner = this,previewView = binding.previewView,analyzer = createImageAnalyzer())}private fun createImageAnalyzer() = ImageAnalysis.Analyzer { imageProxy ->if (isRealTimeDetection) {// 实时检测val bitmap = imageProxy.toBitmap()val results = yoloDetector.detect(bitmap)runOnUiThread {binding.overlayView.setDetectionResults(results)}}imageProxy.close()}private fun setupUI() {// 拍照按钮binding.btnCapture.setOnClickListener {cameraManager.takePhoto(onImageCaptured = { file ->processImage(file)},onError = { exception ->Toast.makeText(this, "拍照失败: ${exception.message}", Toast.LENGTH_SHORT).show()})}// 实时检测开关binding.switchRealTime.setOnCheckedChangeListener { _, isChecked ->isRealTimeDetection = isChecked}// 闪光灯按钮binding.btnFlash.setOnClickListener {cameraManager.toggleFlash()}}private fun processImage(file: File) {lifecycleScope.launch {val bitmap = BitmapFactory.decodeFile(file.absolutePath)val results = yoloDetector.detect(bitmap)// 显示结果val intent = Intent(this@CameraActivity, ResultActivity::class.java)intent.putExtra("image_path", file.absolutePath)intent.putExtra("results", Gson().toJson(results))startActivity(intent)}}private fun allPermissionsGranted() = REQUIRED_PERMISSIONS.all {ContextCompat.checkSelfPermission(baseContext, it) == PackageManager.PERMISSION_GRANTED}private fun requestPermissions() {ActivityCompat.requestPermissions(this,REQUIRED_PERMISSIONS,REQUEST_CODE_PERMISSIONS)}companion object {private const val REQUEST_CODE_PERMISSIONS = 10private val REQUIRED_PERMISSIONS = arrayOf(Manifest.permission.CAMERA,Manifest.permission.WRITE_EXTERNAL_STORAGE)}
}

8. 目标检测实现

8.1 YOLOv11检测器

class YoloDetector(private val context: Context) {private var interpreter: Interpreter? = nullprivate var labels: List<String> = emptyList()private var inputSize = 640private var confThreshold = 0.25fprivate var iouThreshold = 0.45finit {loadModel()loadLabels()loadConfig()}private fun loadModel() {try {val modelFile = loadModelFile("yolov11.tflite")val options = Interpreter.Options()// 使用GPU加速（如果可用)val gpuDelegate = GpuDelegate()options.addDelegate(gpuDelegate)// 或使用NNAPI// options.setUseNNAPI(true)// 设置线程数options.setNumThreads(4)interpreter = Interpreter(modelFile, options)Log.d("YoloDetector", "模型加载成功")} catch (e: Exception) {Log.e("YoloDetector", "模型加载失败", e)}}private fun loadModelFile(filename: String): ByteBuffer {val assetFileDescriptor = context.assets.openFd(filename)val inputStream = FileInputStream(assetFileDescriptor.fileDescriptor)val fileChannel = inputStream.channelval startOffset = assetFileDescriptor.startOffsetval declaredLength = assetFileDescriptor.declaredLengthreturn fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength)}private fun loadLabels() {try {labels = context.assets.open("labels.txt").bufferedReader().readLines()} catch (e: Exception) {Log.e("YoloDetector", "标签加载失败", e)}}private fun loadConfig() {try {val json = context.assets.open("config.json").bufferedReader().readText()val config = Gson().fromJson(json, ModelConfig::class.java)inputSize = config.input_sizeconfThreshold = config.conf_thresholdiouThreshold = config.iou_threshold} catch (e: Exception) {Log.e("YoloDetector", "配置加载失败", e)}}// 主检测方法fun detect(bitmap: Bitmap): List<Detection> {val startTime = System.currentTimeMillis()// 1. 预处理val inputBitmap = Bitmap.createScaledBitmap(bitmap, inputSize, inputSize, true)val inputBuffer = preprocessImage(inputBitmap)// 2. 推理val outputBuffer = runInference(inputBuffer)// 3. 后处理val detections = postprocess(outputBuffer, bitmap.width, bitmap.height)val inferenceTime = System.currentTimeMillis() - startTimeLog.d("YoloDetector", "检测耗时: ${inferenceTime}ms, 检测到: ${detections.size}个对象")return detections}private fun preprocessImage(bitmap: Bitmap): ByteBuffer {val inputBuffer = ByteBuffer.allocateDirect(4 * inputSize * inputSize * 3)inputBuffer.order(ByteOrder.nativeOrder())val intValues = IntArray(inputSize * inputSize)bitmap.getPixels(intValues, 0, bitmap.width, 0, 0, bitmap.width, bitmap.height)var pixel = 0for (i in 0 until inputSize) {for (j in 0 until inputSize) {val value = intValues[pixel++]// 归一化到[0, 1]inputBuffer.putFloat(((value shr 16) and 0xFF) / 255.0f)  // RinputBuffer.putFloat(((value shr 8) and 0xFF) / 255.0f)   // GinputBuffer.putFloat((value and 0xFF) / 255.0f)           // B}}return inputBuffer}private fun runInference(inputBuffer: ByteBuffer): Array<FloatArray> {// YOLOv11输出形状: [1, 84, 8400] (80类 + 4个框坐标)val outputShape = intArrayOf(1, 84, 8400)val outputBuffer = Array(outputShape[0]) {Array(outputShape[1]) {FloatArray(outputShape[2])}}interpreter?.run(inputBuffer, outputBuffer)return outputBuffer[0]}private fun postprocess(output: Array<FloatArray>,originalWidth: Int,originalHeight: Int): List<Detection> {val detections = mutableListOf<Detection>()// output shape: [84, 8400]// 前4行是边界框坐标 (x, y, w, h)// 后80行是类别置信度for (i in 0 until output[0].size) {// 获取最大置信度类别var maxConf = 0fvar maxClassIdx = 0for (classIdx in 4 until output.size) {val conf = output[classIdx][i]if (conf > maxConf) {maxConf = confmaxClassIdx = classIdx - 4}}// 过滤低置信度if (maxConf < confThreshold) continue// 解析边界框val cx = output[0][i] * originalWidth / inputSizeval cy = output[1][i] * originalHeight / inputSizeval w = output[2][i] * originalWidth / inputSizeval h = output[3][i] * originalHeight / inputSizeval left = cx - w / 2val top = cy - h / 2val right = cx + w / 2val bottom = cy + h / 2detections.add(Detection(classIndex = maxClassIdx,className = labels.getOrNull(maxClassIdx) ?: "Unknown",confidence = maxConf,bbox = RectF(left, top, right, bottom)))}// NMS非极大值抑制return nms(detections, iouThreshold)}private fun nms(detections: List<Detection>, iouThreshold: Float): List<Detection> {val sortedDetections = detections.sortedByDescending { it.confidence }val selected = mutableListOf<Detection>()for (detection in sortedDetections) {var shouldSelect = truefor (selectedDetection in selected) {if (detection.classIndex == selectedDetection.classIndex) {val iou = calculateIoU(detection.bbox, selectedDetection.bbox)if (iou > iouThreshold) {shouldSelect = falsebreak}}}if (shouldSelect) {selected.add(detection)}}return selected}private fun calculateIoU(box1: RectF, box2: RectF): Float {val intersectionLeft = maxOf(box1.left, box2.left)val intersectionTop = maxOf(box1.top, box2.top)val intersectionRight = minOf(box1.right, box2.right)val intersectionBottom = minOf(box1.bottom, box2.bottom)val intersectionWidth = maxOf(0f, intersectionRight - intersectionLeft)val intersectionHeight = maxOf(0f, intersectionBottom - intersectionTop)val intersectionArea = intersectionWidth * intersectionHeightval box1Area = (box1.right - box1.left) * (box1.bottom - box1.top)val box2Area = (box2.right - box2.left) * (box2.bottom - box2.top)val unionArea = box1Area + box2Area - intersectionAreareturn if (unionArea > 0) intersectionArea / unionArea else 0f}fun close() {interpreter?.close()interpreter = null}
}// 检测结果数据类
data class Detection(val classIndex: Int,val className: String,val confidence: Float,val bbox: RectF
)data class ModelConfig(val input_size: Int,val conf_threshold: Float,val iou_threshold: Float
)

8.2 结果可视化View

class DetectionOverlayView @JvmOverloads constructor(context: Context,attrs: AttributeSet? = null,defStyleAttr: Int = 0
) : View(context, attrs, defStyleAttr) {private val boxPaint = Paint().apply {style = Paint.Style.STROKEstrokeWidth = 5fcolor = Color.GREEN}private val textPaint = Paint().apply {style = Paint.Style.FILLcolor = Color.WHITEtextSize = 40ftypeface = Typeface.DEFAULT_BOLD}private val backgroundPaint = Paint().apply {style = Paint.Style.FILLcolor = Color.argb(180, 0, 255, 0)}private var detections: List<Detection> = emptyList()fun setDetectionResults(results: List<Detection>) {detections = resultsinvalidate()}override fun onDraw(canvas: Canvas) {super.onDraw(canvas)for (detection in detections) {// 绘制边界框canvas.drawRect(detection.bbox, boxPaint)// 绘制标签背景val label = "${detection.className} ${(detection.confidence * 100).toInt()}%"val textBounds = Rect()textPaint.getTextBounds(label, 0, label.length, textBounds)val labelLeft = detection.bbox.leftval labelTop = detection.bbox.top - textBounds.height() - 10val labelRight = labelLeft + textBounds.width() + 20val labelBottom = detection.bbox.topcanvas.drawRect(labelLeft, labelTop, labelRight, labelBottom, backgroundPaint)// 绘制标签文本canvas.drawText(label, labelLeft + 10, labelBottom - 5, textPaint)}}
}

9. MES系统对接

9.1 MES API接口定义

interface MesApi {// 上传检测结果@Multipart@POST("api/mes/detection/upload")suspend fun uploadDetectionResult(@Part("orderId") orderId: RequestBody,@Part("stationId") stationId: RequestBody,@Part("operatorId") operatorId: RequestBody,@Part("timestamp") timestamp: RequestBody,@Part("detectionData") detectionData: RequestBody,@Part image: MultipartBody.Part): Response<MesResponse>// 获取工单信息@GET("api/mes/order/{orderId}")suspend fun getOrderInfo(@Path("orderId") orderId: String): Response<OrderInfo>// 批量上传@POST("api/mes/detection/batch")suspend fun batchUpload(@Body results: List<DetectionUploadData>): Response<MesResponse>
}// 数据模型
data class MesResponse(val code: Int,val message: String,val data: Any?
)data class OrderInfo(val orderId: String,val productCode: String,val productName: String,val quantity: Int,val status: String
)data class DetectionUploadData(val orderId: String,val stationId: String,val operatorId: String,val timestamp: Long,val imagePath: String,val detections: List<DetectionData>
)data class DetectionData(val className: String,val confidence: Float,val x: Float,val y: Float,val width: Float,val height: Float
)

9.2 MES同步服务

@HiltViewModel
class MesSyncViewModel @Inject constructor(private val mesApi: MesApi,private val detectionDao: DetectionDao
) : ViewModel() {private val _syncState = MutableStateFlow<SyncState>(SyncState.Idle)val syncState: StateFlow<SyncState> = _syncState.asStateFlow()// 上传单个检测结果fun uploadResult(result: DetectionResult,orderId: String,stationId: String,operatorId: String) {viewModelScope.launch {_syncState.value = SyncState.Uploadingtry {val imageFile = File(result.imagePath)val imagePart = MultipartBody.Part.createFormData("image",imageFile.name,imageFile.asRequestBody("image/jpeg".toMediaTypeOrNull()))val detectionData = Gson().toJson(mapOf("detections" to Gson().fromJson(result.detections,Array<Detection>::class.java).map {DetectionData(className = it.className,confidence = it.confidence,x = it.bbox.left,y = it.bbox.top,width = it.bbox.width(),height = it.bbox.height())}))val response = mesApi.uploadDetectionResult(orderId = orderId.toRequestBody(),stationId = stationId.toRequestBody(),operatorId = operatorId.toRequestBody(),timestamp = result.timestamp.toString().toRequestBody(),detectionData = detectionData.toRequestBody(),image = imagePart)if (response.isSuccessful && response.body()?.code == 200) {// 标记为已同步detectionDao.markAsSynced(result.id)_syncState.value = SyncState.Success("上传成功")} else {_syncState.value = SyncState.Error("上传失败: ${response.body()?.message}")}} catch (e: Exception) {_syncState.value = SyncState.Error("上传异常: ${e.message}")Log.e("MesSyncViewModel", "上传失败", e)}}}// 批量同步未上传的结果fun syncUnsyncedResults() {viewModelScope.launch {detectionDao.getUnsyncedResults().collect { unsyncedList ->if (unsyncedList.isEmpty()) {_syncState.value = SyncState.Success("没有待同步数据")return@collect}_syncState.value = SyncState.Uploadingtry {val uploadDataList = unsyncedList.map { result ->DetectionUploadData(orderId = result.mesOrderId ?: "",stationId = "STATION_001", // 从配置获取operatorId = "OPERATOR_001", // 从配置获取timestamp = result.timestamp,imagePath = result.imagePath,detections = Gson().fromJson(result.detections,Array<Detection>::class.java).map {DetectionData(className = it.className,confidence = it.confidence,x = it.bbox.left,y = it.bbox.top,width = it.bbox.width(),height = it.bbox.height())})}val response = mesApi.batchUpload(uploadDataList)if (response.isSuccessful && response.body()?.code == 200) {// 标记所有为已同步unsyncedList.forEach { result ->detectionDao.markAsSynced(result.id)}_syncState.value = SyncState.Success("批量同步成功")} else {_syncState.value = SyncState.Error("批量同步失败")}} catch (e: Exception) {_syncState.value = SyncState.Error("同步异常: ${e.message}")}}}}private fun String.toRequestBody(): RequestBody {return this.toRequestBody("text/plain".toMediaTypeOrNull())}
}sealed class SyncState {object Idle : SyncState()object Uploading : SyncState()data class Success(val message: String) : SyncState()data class Error(val message: String) : SyncState()
}

9.3 后台同步Service

class SyncService : Service() {@Injectlateinit var mesApi: MesApi@Injectlateinit var detectionDao: DetectionDaoprivate val serviceScope = CoroutineScope(SupervisorJob() + Dispatchers.IO)override fun onBind(intent: Intent?): IBinder? = nulloverride fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {// 创建前台通知createNotificationChannel()val notification = createNotification()startForeground(NOTIFICATION_ID, notification)// 开始同步startSync()return START_STICKY}private fun startSync() {serviceScope.launch {detectionDao.getUnsyncedResults().collect { unsyncedList ->if (unsyncedList.isNotEmpty()) {syncResults(unsyncedList)}delay(60000) // 每分钟检查一次}}}private suspend fun syncResults(results: List<DetectionResult>) {for (result in results) {try {// 上传逻辑（简化版）val uploadData = DetectionUploadData(orderId = result.mesOrderId ?: "",stationId = "STATION_001",operatorId = "OPERATOR_001",timestamp = result.timestamp,imagePath = result.imagePath,detections = emptyList() // 实际应解析)val response = mesApi.batchUpload(listOf(uploadData))if (response.isSuccessful) {detectionDao.markAsSynced(result.id)}} catch (e: Exception) {Log.e("SyncService", "同步失败", e)}}}private fun createNotificationChannel() {if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {val channel = NotificationChannel(CHANNEL_ID,"数据同步",NotificationManager.IMPORTANCE_LOW)val notificationManager = getSystemService(NotificationManager::class.java)notificationManager.createNotificationChannel(channel)}}private fun createNotification(): Notification {return NotificationCompat.Builder(this, CHANNEL_ID).setContentTitle("数据同步中").setContentText("正在后台同步检测结果到MES系统").setSmallIcon(R.drawable.ic_sync).setPriority(NotificationCompat.PRIORITY_LOW).build()}override fun onDestroy() {super.onDestroy()serviceScope.cancel()}companion object {private const val CHANNEL_ID = "sync_channel"private const val NOTIFICATION_ID = 1001}
}

10. 性能优化

10.1 模型优化

// 1. 使用GPU加速
class OptimizedYoloDetector(context: Context) : YoloDetector(context) {override fun loadModel() {val options = Interpreter.Options()// GPU委托val gpuDelegate = GpuDelegate(GpuDelegate.Options().apply {setPrecisionLossAllowed(true) // 允许精度损失以提升速度setInferencePreference(GpuDelegate.Options.INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER)})options.addDelegate(gpuDelegate)// XNNPACK委托（CPU优化）val xnnpackDelegate = XNNPackDelegate()options.addDelegate(xnnpackDelegate)interpreter = Interpreter(modelFile, options)}
}// 2. 图像预处理优化
class FastPreprocessor {private val intValues = IntArray(640 * 640)private val floatValues = FloatArray(640 * 640 * 3)fun preprocess(bitmap: Bitmap): ByteBuffer {// 使用对象池复用ByteBufferval buffer = ByteBuffer.allocateDirect(4 * 640 * 640 * 3)buffer.order(ByteOrder.nativeOrder())bitmap.getPixels(intValues, 0, bitmap.width, 0, 0, bitmap.width, bitmap.height)// 并行处理(0 until 640 * 640).toList().parallelStream().forEach { i ->val pixelValue = intValues[i]floatValues[i * 3] = ((pixelValue shr 16) and 0xFF) / 255.0ffloatValues[i * 3 + 1] = ((pixelValue shr 8) and 0xFF) / 255.0ffloatValues[i * 3 + 2] = (pixelValue and 0xFF) / 255.0f}buffer.asFloatBuffer().put(floatValues)return buffer}
}

10.2 内存优化

class MemoryManager {// Bitmap对象池private val bitmapPool = object : LruCache<String, Bitmap>(20) {override fun sizeOf(key: String, value: Bitmap): Int {return value.byteCount / 1024}override fun entryRemoved(evicted: Boolean,key: String,oldValue: Bitmap,newValue: Bitmap?) {if (!oldValue.isRecycled) {oldValue.recycle()}}}fun getBitmap(key: String, width: Int, height: Int): Bitmap {return bitmapPool.get(key) ?: Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888).also { bitmapPool.put(key, it) }}// 定期清理fun trimMemory() {bitmapPool.trimToSize(bitmapPool.size() / 2)System.gc()}
}

10.3 网络优化

class OptimizedNetworkModule {@Provides@Singletonfun provideOkHttpClient(): OkHttpClient {return OkHttpClient.Builder()// 连接池.connectionPool(ConnectionPool(5, 5, TimeUnit.MINUTES))// 缓存.cache(Cache(File(context.cacheDir, "http_cache"), 50L * 1024 * 1024))// 重试.retryOnConnectionFailure(true)// 超时.connectTimeout(15, TimeUnit.SECONDS).readTimeout(30, TimeUnit.SECONDS).writeTimeout(30, TimeUnit.SECONDS)// 拦截器.addInterceptor(CacheInterceptor()).addInterceptor(RetryInterceptor(3)).build()}
}// 缓存拦截器
class CacheInterceptor : Interceptor {override fun intercept(chain: Interceptor.Chain): okhttp3.Response {val request = chain.request()val response = chain.proceed(request)return response.newBuilder().header("Cache-Control", "public, max-age=60").build()}
}// 重试拦截器
class RetryInterceptor(private val maxRetry: Int) : Interceptor {override fun intercept(chain: Interceptor.Chain): okhttp3.Response {val request = chain.request()var response = chain.proceed(request)var tryCount = 0while (!response.isSuccessful && tryCount < maxRetry) {tryCount++Thread.sleep(1000 * tryCount.toLong())response.close()response = chain.proceed(request)}return response}
}

11. 部署与测试

11.1 打包配置

android {signingConfigs {release {storeFile file('../keystore/release.jks')storePassword 'your_store_password'keyAlias 'your_key_alias'keyPassword 'your_key_password'}}buildTypes {release {minifyEnabled trueshrinkResources trueproguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'signingConfig signingConfigs.release}}// 多渠道打包flavorDimensions "version"productFlavors {production {dimension "version"buildConfigField "String", "BASE_URL", '"https://api.production.com"'}staging {dimension "version"buildConfigField "String", "BASE_URL", '"https://api.staging.com"'}}
}

11.2 ProGuard规则

# proguard-rules.pro# 保留TensorFlow Lite
-keep class org.tensorflow.** { *; }
-keep interface org.tensorflow.** { *; }# 保留ONNX Runtime
-keep class ai.onnxruntime.** { *; }# 保留数据类
-keep class com.example.yolodetector.data.** { *; }
-keep class com.example.yolodetector.domain.** { *; }# 保留Retrofit
-keepattributes Signature
-keepattributes *Annotation*
-keep class retrofit2.** { *; }# 保留Gson
-keep class com.google.gson.** { *; }
-keep class * implements com.google.gson.TypeAdapter
-keep class * implements com.google.gson.TypeAdapterFactory
-keep class * implements com.google.gson.JsonSerializer
-keep class * implements com.google.gson.JsonDeserializer

11.3 单元测试

@RunWith(AndroidJUnit4::class)
class YoloDetectorTest {private lateinit var detector: YoloDetectorprivate lateinit var context: Context@Beforefun setup() {context = ApplicationProvider.getApplicationContext()detector = YoloDetector(context)}@Testfun testModelLoading() {assertNotNull(detector)}@Testfun testDetection() {// 创建测试图片val testBitmap = Bitmap.createBitmap(640, 640, Bitmap.Config.ARGB_8888)val canvas = Canvas(testBitmap)canvas.drawColor(Color.WHITE)// 执行检测val results = detector.detect(testBitmap)assertNotNull(results)assertTrue(results is List)}@Testfun testNMS() {val detections = listOf(Detection(0, "person", 0.9f, RectF(10f, 10f, 100f, 100f)),Detection(0, "person", 0.8f, RectF(15f, 15f, 105f, 105f)),Detection(1, "car", 0.7f, RectF(200f, 200f, 300f, 300f)))// 测试NMS逻辑// ...}@Afterfun tearDown() {detector.close()}
}

11.4 性能测试

@RunWith(AndroidJUnit4::class)
@LargeTest
class PerformanceTest {@Testfun testInferenceTime() {val detector = YoloDetector(ApplicationProvider.getApplicationContext())val testBitmap = BitmapFactory.decodeResource(ApplicationProvider.getApplicationContext<Context>().resources,R.drawable.test_image)val times = mutableListOf<Long>()// 预热repeat(5) {detector.detect(testBitmap)}// 测试repeat(100) {val start = System.currentTimeMillis()detector.detect(testBitmap)val end = System.currentTimeMillis()times.add(end - start)}val avgTime = times.average()val maxTime = times.maxOrNull() ?: 0Lval minTime = times.minOrNull() ?: 0LLog.d("PerformanceTest", "平均推理时间: ${avgTime}ms")Log.d("PerformanceTest", "最大推理时间: ${maxTime}ms")Log.d("PerformanceTest", "最小推理时间: ${minTime}ms")// 断言平均时间应该在合理范围内assertTrue("推理时间过长", avgTime < 500)}
}

12. 常见问题与解决方案

12.1 模型加载问题

问题: 模型加载失败或内存溢出

解决方案:

// 1. 使用模型量化
// 2. 延迟加载
class LazyYoloDetector(private val context: Context) {private val detector: YoloDetector by lazy {YoloDetector(context)}fun detect(bitmap: Bitmap) = detector.detect(bitmap)
}// 3. 检查模型文件完整性
fun verifyModelFile(context: Context): Boolean {return try {val fd = context.assets.openFd("yolov11.tflite")fd.length > 0} catch (e: Exception) {false}
}

12.2 相机权限问题

问题: Android 10+存储权限变化

解决方案:

// 使用Scoped Storage
fun saveImageToGallery(context: Context, bitmap: Bitmap) {val contentValues = ContentValues().apply {put(MediaStore.Images.Media.DISPLAY_NAME, "IMG_${System.currentTimeMillis()}.jpg")put(MediaStore.Images.Media.MIME_TYPE, "image/jpeg")put(MediaStore.Images.Media.RELATIVE_PATH, Environment.DIRECTORY_PICTURES)}val uri = context.contentResolver.insert(MediaStore.Images.Media.EXTERNAL_CONTENT_URI,contentValues)uri?.let {context.contentResolver.openOutputStream(it)?.use { outputStream ->bitmap.compress(Bitmap.CompressFormat.JPEG, 95, outputStream)}}
}

12.3 网络连接问题

问题: MES系统连接不稳定

解决方案:

// 实现离线队列
class OfflineQueueManager(private val dao: DetectionDao,private val workManager: WorkManager
) {fun enqueueUpload(result: DetectionResult) {val uploadWork = OneTimeWorkRequestBuilder<UploadWorker>().setConstraints(Constraints.Builder().setRequiredNetworkType(NetworkType.CONNECTED).build()).setBackoffCriteria(BackoffPolicy.EXPONENTIAL,WorkRequest.MIN_BACKOFF_MILLIS,TimeUnit.MILLISECONDS).setInputData(workDataOf("result_id" to result.id)).build()workManager.enqueue(uploadWork)}
}class UploadWorker(context: Context,params: WorkerParameters
) : CoroutineWorker(context, params) {override suspend fun doWork(): Result {val resultId = inputData.getLong("result_id", -1)// 上传逻辑return try {// upload...Result.success()} catch (e: Exception) {Result.retry()}}
}

12.4 性能优化建议

// 1. 使用协程优化
class OptimizedDetectionViewModel @Inject constructor(private val detector: YoloDetector
) : ViewModel() {private val detectionScope = CoroutineScope(Dispatchers.Default + SupervisorJob())fun detectAsync(bitmap: Bitmap, callback: (List<Detection>) -> Unit) {detectionScope.launch {val results = detector.detect(bitmap)withContext(Dispatchers.Main) {callback(results)}}}
}// 2. 图像降采样
fun downscaleImage(bitmap: Bitmap, maxSize: Int): Bitmap {val ratio = maxSize.toFloat() / maxOf(bitmap.width, bitmap.height)if (ratio >= 1) return bitmapval newWidth = (bitmap.width * ratio).toInt()val newHeight = (bitmap.height * ratio).toInt()return Bitmap.createScaledBitmap(bitmap, newWidth, newHeight, true)
}// 3. 使用RenderScript加速(已弃用，改用Vulkan或GPU Compute)