1

In my project of remote assistence for smartphones android, i'm trying simulate touch on screen (using AccessibilityService) based in coordinates of mouse inside a Image component (Delphi), that is where i receives the screen of remote device.

enter image description here

Image component is inside a ScrollBox component

  • Image component is: Align: alNone, AutoSize: True
  • ScrollBox is: Align: alClient, AutoScroll: True, AutoSize: False

Code of redimension of Image component:

procedure TForm2.Checkbox1Click(Sender: TObject);
begin
  if Checkbox1.Checked then
  begin
    Image1.AutoSize := false;
    Image1.Stretch := true;
    Image1.Align := alClient;
  end
  else
  begin
    Image1.AutoSize := true;
    Image1.Stretch := false;
    Image1.Align := alNone;
  end;
end;

This is how i send coordinates:

procedure TForm2.Image1MouseDown(Sender: TObject; Button: TMouseButton;
  Shift: TShiftState; X, Y: Integer);
begin

  X := (X * Image1.Width) div Image1.Width;
  Y := (Y * Image1.Height) div Image1.Height;

  SS1.Socket.Connections[Index].SendText('touch' + IntToStr(X) +
    '<|>' + IntToStr(Y) + #13#10);
end;

Then i tried this on Java code (android):

String xline;

if (xline.contains("touch")) {

    String coordinates = xline.replace("touch", "");

    String[] tokens = coordinates.split(Pattern.quote("<|>"));

    float x = parseFloat(tokens[0]);
    float y = parseFloat(tokens[1]);

    int screenWidth = getResources().getDisplayMetrics().widthPixels;
    int screenHeight = getResources().getDisplayMetrics().heightPixels;

    x = (x * screenWidth) / screenWidth;
    y = (y * screenHeight) / screenHeight;

    // touch on screen with x and y
}

but the touch is happens in a place very distant.

How fix this?

  • Note that these operations don't change X and Y: `X := (X * Image1.Width) div Image1.Width;`. And what is desired result? Coordinates of what? – MBo Aug 20 '18 at 06:31
  • @MBo, i want that for example: when i click on *WhatsApp* icon, the touch on smartphone also be on whatsapp icon. This is the result wished. –  Aug 20 '18 at 09:50
  • Then, no one know say to me how calc these coordinates to touch on screen of smartphone in same place where i point with mouse on `TImage` component (Delphi application screenshot above)? –  Aug 20 '18 at 22:57

1 Answers1

1

The solution was found and the reference was this answer:

procedure TForm2.Image1MouseDown(Sender: TObject; Button: TMouseButton;
  Shift: TShiftState; X, Y: Integer);
var
  Index, XTouch, YTouch, RXCoord, RYCoord: Integer;
  List: TStrings;
  RScreen: String;
begin
  Index := Form1.ListView1.ItemIndex;
  if Index = -1 then
    Exit;

  List := TStringList.Create;
  RScreen := Form1.ListView1.Selected.SubItems[6];

  try
    ExtractStrings(['x'], [], PChar(RScreen), List); // Ex: my smartphone is 1920x1080
    RYCoord := StrToInt(List[0]); // 1920 (height)
    RXCoord := StrToInt(List[1]); // 1080 (width)
  finally
    List.Free;
  end;

  XTouch := Round((X / Image1.Width) * RXCoord);
  YTouch := Round((Y / Image1.Height) * RYCoord);

  Form1.SS1.Socket.Connections[Index].SendText('touch' + IntToStr(XTouch)
    + '<|>' + IntToStr(YTouch) + #13#10);
end;