DETAILED ACTION
Response to Amendment
This Action is responsive to Applicant’s response filed on 06/11/2026. All claims are still pending
in the present application. This Action is made FINAL.
Amendment
Applicant submitted amendments on 06/11/2026. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Response to Arguments
In regards to the rejection of claim(s) 1-20 under 35 U.S.C. 101 is withdrawn in view of the amendments to independent claim(s) 1 and 20. The amended claims recite additional limitations that integrate the recited data analysis into a practical application by generating and transmitting an adjustment command to one or more terminal devices to cause adjustment of a display region according to the analyzed joint-motion information. Accordingly, the claims are no longer considered directed to patent ineligible subject matter under 35 U.S.C. 101.
In regards to argument(s), Applicant(s) state(s) that, in rejecting the technical features highlighted above, primarily relies on Liu and Zhang. However, neither reference, alone or in combination, discloses or renders obvious the claimed mechanism of "calculate a first score corresponding to the first joint according to a first displacement of the first joint; in response to the first score being greater than a threshold, generate an adjustment command for adjusting a first display region of the first image according to the score of the first joint" as recited the currently presented claim 1, therefore, the rejection of 35 U.S.C. 103 should be removed, (Emphasis added, Remarks, page 12-15).
Applicant’s arguments have been considered but are moot in view of the new ground(s) of rejection in view of Liu et al (CN 117520577 A) in view of Zhang (CN 115830697 A), further in view of Fang et al (US 2024/0223881 A1).
Office Action Summary
Claim(s) 1-8, 14-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (CN 117520577 A; See translation provided by Examiner) in view of Zhang (CN 115830697 A; See translation provided by Examiner), further in view of Fang et al (US 2024/0223881 A1).
Claim(s) 9-12 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (CN 117520577 A; See translation provided by Examiner) in view of Zhang (CN 115830697 A; See translation provided by Examiner) and Fang et al (US 2024/0223881 A1), further in view of Cui et al (US 2024/0314268 A1).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (CN 117520577 A; See translation provided by Examiner) in view of Zhang (CN 115830697 A; See translation provided by Examiner), Fang et al (US 2024/0223881 A1), and Cui et al (US 2024/0314268 A1), further in view of Zhang et al (CN 110045823 A; See translation provided by Examiner).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 14-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (CN 117520577 A; See translation provided by Examiner) in view of Zhang (CN 115830697 A; See translation provided by Examiner), further in view of Fang et al (US 2024/0223881 A1).
Regarding claim(s) 1 and 20, Liu teaches an electronic device for adjusting a display region based on motion, comprising:
a transceiver, obtaining a first image comprising a first target object (Figure 11; and Paragraph [0025]: “This information can be acquired through an image acquisition device and included in the image information acquired by the device”), wherein the electronic device is communicatively connected to one or more terminal devices through the transceiver (Paragraph [0026]: “After uploading this information to a server and processing it, the movement data of the virtual image corresponding to the user's movement can be obtained”; Paragraph [0028]: “They send the acquired data to the server, which processes the data and obtains accurate three-dimensional motion information”; and Paragraph [0097]: “The server may send the stored program instructions to other devices for execution, or it may execute the stored program instructions itself”); and
a processor, coupled to the transceiver (Figure 9; and Paragraph [0092]), configured to:
detect the first image to obtain a first joint of the first target object (Paragraph [0036]: “obtaining the user's motion information, key points of several movements of the user can be determined from the motion information. These key points are the key connection points when the user moves, such as elbows, wrists, shoulders, waist, hips, knees, ankles”);
transmit the adjustment command to the one or more terminal devices through the transceiver to instruct the one or more terminal devices to adjust the first display region of the first image according to the adjustment command (Paragraph [0097]: “The server may send the stored program instructions to other devices for execution, or it may execute the stored program instructions itself”).
Liu fails to teach calculate a first score corresponding to the first joint according to a first displacement of the first joint; and in response to the first score being greater than a threshold, generate an adjustment command for adjusting a first display region of the first image according to the score of the first joint. However, Zhang (CN, “697”) teaches calculate a first score corresponding to the first joint according to a first displacement of the first joint (Paragraph [0075]: “Compute the displacement vectors of keypoints in the first and last frames of the continuous multi-frame classroom image”; and Paragraph [0076]: “Calculate the ratio of the mean value of the modulus of each displacement vector to the width of the detection frame”);
in response to the first score being greater than a threshold(Paragraph [0077] – Paragraph [0079]: “The calculated ratio is compared with a preset threshold, and if the ratio is greater than the preset threshold, it is determined that the second determination result is that the student has a posture change […] When the second determination result is that the student has a posture change, determine multiple frames of classroom images within a preset range centered on the target frame classroom image where the posture change occurs as the target image […] if the second determination result is that the student has a posture change, it is determined that the target frame of the classroom image with the posture change as the center and multiple frames of classroom images within a preset range are used as the target image”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Liu and Zhang (CN, “697”) before the effective filing date of the claimed invention. The motivation for this combination of references would have been to improve the accuracy and reliability of motion evaluation by utilizing displacement information of detected human-body key points across consecutive image frames, as taught by Zhang (CN, “697”), within Liu’s system for evaluating motion attributes associated with detected key points, thereby providing a more accurate determination of changes in a user’s movement. This motivation for the combination of Liu and Zhang (CN, “697”) is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Liu and Zhang (CN, “697”) fails to teach generate an adjustment command for adjusting a first display region of the first image according to the score of the first joint. However, Fang teaches in response to the first score being greater than a threshold, generate an adjustment command for adjusting a first display region of the first image according to the score of the first joint (Paragraph [0028]: “[…] compute a coordinate variation of the identification feature, and utilize the coordinate variation to compute an area variation of the cropping area, so as to adjust the cropping area in accordance with the area variation for conforming to target resolution”).
transmit the adjustment command to the one or more terminal devices through the transceiver to instruct the one or more terminal devices to adjust the first display region of the first image according to the adjustment command (Paragraph [0017]: “the operation processor 16 can utilize an auto framing function of the present application to analyze and adjust the detection image I1, and display an analysis and adjustment result on the screen 14”; and Paragraph [0028]: “[…] compute an area variation of the cropping area, so as to adjust the cropping area in accordance with the area variation for conforming to target resolution”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Liu as modified by Zhang (CN, “697”) with Fang before the effective filing date of the claimed invention. The motivation for this combination of references would have been to apply the detected key-point motion information and displacement-based determination of significant movement provided by Liu and Zhang (CN, “697”) to dynamically adjust a display region based on movement of a detected feature, as taught by Fang, in order to enhance visualization of the moving subject and provide more effective visual feedback, yielding predictable and improved results. This motivation for the combination of Liu, Zhang (CN, “697”), and Fang is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 2, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 1, where Zhang (CN, “697”) teaches wherein the processor (Paragraph [0137]) is further configured to:
calculate a second score according to a second displacement of a second joint of the first target object (Paragraph [0075]: “Compute the displacement vectors of keypoints in the first and last frames of the continuous multi-frame classroom image. Specifically, for the same student, calculate the displacement vector of the key point representing the left eye, the displacement vector of the key point representing the right eye, and the displacement of the key point representing the nose in the first frame and the last frame of the continuous multi-frame classroom image vector, the displacement vector representing the keypoint of the left shoulder, the displacement vector of the keypoint representing the right shoulder, and the displacement vector representing the neck”; and Paragraph [0076]: “Calculate the ratio of the mean value of the modulus of each displacement vector to the width of the detection frame. Specifically, the mean value of the modes of the above six displacement vectors is calculated, and then the ratio of the above mean value to the width of the detection frame is calculated”); and
in response to the second score being greater than the threshold (Paragraph [0077]: “The calculated ratio is compared with a preset threshold, and if the ratio is greater than the preset threshold, it is determined that the second determination result is that the student has a posture change”), and where Fang teaches generate the adjustment command according to the first joint and the second joint (Figure 4A-4B; and Paragraph [0025]: “[…] If the number of the identification feature is single, step S124 and step S126 can be executed to draw the marking frame F by containing the identification feature, and extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and the cropping area Ac can be scaled up in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode. If the number of the identification feature is plural, step S128 and step S130 can be executed to find out one or some nearest objects within the visual field for drawing the marking frame F, and then extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and scale up the cropping area Ac in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode”).
Regarding claim(s) 3, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 2, where Fang teaches wherein the processor (Figure 1; and Paragraph [0017]) is further configured to:
divide a region of interest (read as “cropping area”) of the first image to obtain a plurality of sub-regions (read as “marking frame”), wherein the sub-regions comprise a first sub-region and a second sub-region (Figure 4A-4B; and Paragraph [0025]: “[…] If the number of the identification feature is single, step S124 and step S126 can be executed to draw the marking frame F by containing the identification feature, and extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and the cropping area Ac can be scaled up in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode. If the number of the identification feature is plural, step S128 and step S130 can be executed to find out one or some nearest objects within the visual field for drawing the marking frame F, and then extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and scale up the cropping area Ac in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode”);
where Zhang (CN, “697”) teaches determine that the first joint is located in the first sub-region and the second joint is located in the second sub-region according to the first image (Paragraph [0054]: “Detect six key points including left eye, right eye, nose, left shoulder, right shoulder and neck from the area within the detection frame”; and Paragraph [0056]: “use the trained key point detection model to detect the key points of the human body in the area in the detection frame detected in the previous target detection step, and detect the key points including the left eye, right eye, nose, left shoulder, and right shoulder. and 6 key points of the neck”); and
where Fang teaches in response to the second sub-region being adjacent to the first sub-region (read as “extend the marking”), and where generate the adjustment command according to the first joint and the second joint (Figure 4A-4B; and Paragraph [0025]: “[…] If the number of the identification feature is single, step S124 and step S126 can be executed to draw the marking frame F by containing the identification feature, and extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and the cropping area Ac can be scaled up in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode. If the number of the identification feature is plural, step S128 and step S130 can be executed to find out one or some nearest objects within the visual field for drawing the marking frame F, and then extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and scale up the cropping area Ac in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode”).
Regarding claim(s) 4, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 3, wherein the sub-regions comprise a third sub-region corresponding to a third joint of the first target object (where Zhang (CN, “697”) teaches in Paragraph [0054]: “Detect six key points including left eye, right eye, nose, left shoulder, right shoulder and neck from the area within the detection frame”; and Paragraph [0056]: “use the trained key point detection model to detect the key points of the human body in the area in the detection frame detected in the previous target detection step, and detect the key points including the left eye, right eye, nose, left shoulder, and right shoulder. and 6 key points of the neck”; and where Fang teaches in Paragraph [0025]: “[…] draw the marking frame F by containing the identification feature, and extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area […]”), where Zhang (CN, “697”) teaches wherein a third score corresponding to the third joint is greater than the threshold (Paragraph [0073]: “when the displacement of the key point of the student is greater than a preset threshold, it is considered that the second determination result is that the student has a posture change”; Paragraph [0075]: “Compute the displacement vectors of keypoints in the first and last frames of the continuous multi-frame classroom image”; and Paragraph [0077]: “The calculated ratio is compared with a preset threshold, and if the ratio is greater than the preset threshold, it is determined that the second determination result is that the student has a posture change”), and the processor (Paragraph [0137]) is further configured to:
where Fang teaches in response to the first sub-region being adjacent to the second sub-region and the third sub-region (read as “extend the marking”) being adjacent to at least one of the first sub-region and the second sub-region, generate the adjustment command according to the first joint, the second joint, and the third joint (Figure 4A-4B; Paragraph [0023]: “The marking frame F can contain one or several identification features […] The marking frame F can be drawn by a number and relative positions of the identification features”; Paragraph [0024]: “the identification features between the objects O1 and O2 shown in FIG. 8 are distant from each other […] The embodiment in FIG. 9 can show a situation of the object O2 moved close to the object O1 for preferred performance.”; and Paragraph [0025]: “[…] If the number of the identification feature is plural, step S128 and step S130 can be executed to find out one or some nearest objects within the visual field for drawing the marking frame F, and then extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac […]”).
Regarding claim(s) 5, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 3, where Fang teaches wherein the processor (Figure 1; and Paragraph [0017]) is further configured to:
generate the first display region according to at least one sub-region comprising the first sub-region (Figure 4A-4B; Paragraph [0023]: “The marking frame F can contain one or several identification features […] The marking frame F can be drawn by a number and relative positions of the identification features, and then be extended to conform to the preset aspect ratio (about 16:9 or 4:3) of the screen 14 for generating the cropping area Ac”; and Paragraph [0025]: “[…] If the number of the identification feature is plural, step S128 and step S130 can be executed to find out one or some nearest objects within the visual field for drawing the marking frame F, and then extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac […]”); and
in response to the second sub-region not being adjacent to the at least one sub-region, generate a second display region of the first image according to the second sub-region (Paragraph [0024]: “the identification features between the objects O1 and O2 shown in FIG. 8 are distant from each other, so that the marking frame F and the cropping area Ac may have the wider range and a zooming effect is not obvious”; Paragraph [0019]: “When the object O2 moves, the auto framing method of the present application can still adjust the visual field by moving the middle point or the specific computed point of the foresaid two identification features on the frame center of the screen 14, or adjust the visual field only based on the identification feature of the object O2 immediately”; and Paragraph [0025]: “[…] If the number of the identification feature is plural, step S128 and step S130 can be executed to find out one or some nearest objects within the visual field for drawing the marking frame F”).
Regarding claim(s) 6, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 5, where Fang teaches wherein the adjustment command instructs that the first display region of the first image is output in a first period and the second display region of the first image is output in a second period (Paragraph [0008]: “setting the preset step count in accordance with an animation time of the cropping area, and/or setting the preset frame interval in accordance with an animation speed of the cropping area”; and Paragraph [0028]: “[…] compute a coordinate variation of the identification feature, and utilize the coordinate variation to compute an area variation of the cropping area, so as to adjust the cropping area in accordance with the area variation for conforming to target resolution. With the movement of the object O1 or O2, the image receiver 12 can continuously acquire or capture the plurality of detection images […] so as to determine whether to scale the cropping area Ac or to scale up the cropping area Ac to the target resolution for being displayed on the screen 14 to form the auto framing animation”).
Regarding claim(s) 7, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 6, where Fang teaches wherein the processor (Figure 1; and Paragraph [0017]) is further configured to:
in response to a size of the second display region (read as “cropping area”) being different from a size of the first display region, generate the adjustment command comprising a zoom (read as “scaled image”) in operation or a zoom out operation (Paragraph [0028]: “[…] compute a coordinate variation of the identification feature, and utilize the coordinate variation to compute an area variation of the cropping area, so as to adjust the cropping area in accordance with the area variation for conforming to target resolution. With the movement of the object O1 or O2, the image receiver 12 can continuously acquire or capture the plurality of detection images […] so as to determine whether to scale the cropping area Ac or to scale up the cropping area Ac to the target resolution for being displayed on the screen 14 to form the auto framing animation”).
Regarding claim(s) 8, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 3, where Fang teaches wherein the processor is further configured to:
determine a boundary of the first display region (read as “cropping area Ac”) according to a union of the first sub-region and the second sub-region (Figure 4A-4B; and Paragraph [0025]: “[…] If the number of the identification feature is single, step S124 and step S126 can be executed to draw the marking frame F by containing the identification feature, and extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and the cropping area Ac can be scaled up in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode. If the number of the identification feature is plural, step S128 and step S130 can be executed to find out one or some nearest objects within the visual field for drawing the marking frame F, and then extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and scale up the cropping area Ac in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode”).
Regarding claim(s) 14, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 1, where Liu teaches wherein the processor (Figure 9; and Paragraph [0092]) is further configured to:
perform facial recognition on a second target object in a second image to obtain a plurality of joints (Paragraph [0027]: “the facial information of the users can be obtained first, and then the corresponding action information can be obtained. The association between facial information and motion information can be determined by their relative positions [...]”; and Paragraph [0036]: “obtaining the user's motion information, key points of several movements of the user can be determined from the motion information. These key points are the key connection points when the user moves, such as elbows, wrists, shoulders, waist, hips, knees, ankles”);
where Fang teaches calculate an area change of a polygon formed by the joints (Paragraph [0028]: “compute a coordinate variation of the identification feature, and utilize the coordinate variation to compute an area variation of the cropping area […] Difference between the coordinates of the identification feature on each detection image and the coordinates of the identification feature on the previous detection image (or another target image) can be interpreted as the coordinate variation of the identification feature […] the cropping area Ac generated by each detection image may have dimensional and/or shaped difference relative to the cropping area Ac generated by the previous detection image (or another target image), which can be interpreted as the area variation of the cropping area”); and
in response to an absolute value of the area change being greater than a change threshold, generate a playback control command for playback speed control (Paragraph [0026]: “the predefined difference condition may be set as ten percentages”; and Paragraph [0027]: “ If the position difference conforms to the predefined difference condition, step S134 can be executed to set a preset step count and a preset frame interval […] The preset frame interval can be interpreted as time difference between one analyzed frame and another analyzed frame, and can be used to adjust a playback speed of the auto framing animation”).
Regarding claim(s) 15, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 14, where Fang teaches wherein the playback control command is used to reduce a playback speed of the first image (Paragraph [0027]: “ The preset frame interval can be interpreted as time difference between one analyzed frame and another analyzed frame, and can be used to adjust a playback speed of the auto framing animation; an animation speed of the cropping area can be interpreted as a movement speed from the initial detection to the stationary location of the object, so that the preset frame interval can be optionally set in accordance with the animation speed of the cropping area”).
Regarding claim(s) 16, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 1, where Liu teaches wherein the processor (Figure 9; and Paragraph [0092]) is further configured to:
perform facial recognition on a second target object in a second image to obtain a plurality of joints (Paragraph [0027]: “the facial information of the users can be obtained first, and then the corresponding action information can be obtained. The association between facial information and motion information can be determined by their relative positions [...]”; and Paragraph [0036]: “obtaining the user's motion information, key points of several movements of the user can be determined from the motion information. These key points are the key connection points when the user moves, such as elbows, wrists, shoulders, waist, hips, knees, ankles”);
where Fang teaches calculate an area change of a polygon formed by the joints (Paragraph [0028]: “compute a coordinate variation of the identification feature, and utilize the coordinate variation to compute an area variation of the cropping area, so as to adjust the cropping area in accordance with the area variation for conforming to target resolution”); and
generate a playback control command for playback speed control according to the area change (Paragraph [0028]: “utilize the coordinate variation to compute an area variation of the cropping area, so as to adjust the cropping area in accordance with the area variation”; and Paragraph [0027]: “The preset frame interval can be interpreted as time difference between one analyzed frame and another analyzed frame, and can be used to adjust a playback speed of the auto framing animation”).
Regarding claim(s) 17, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 16, where Zhang (CN, “697”) teaches wherein the processor (Paragraph [00137]) is further configured to:
in response to the area change being greater than a first threshold (Paragraph [0077] – Paragraph [0079]: “The calculated ratio is compared with a preset threshold, and if the ratio is greater than the preset threshold, it is determined that the second determination result is that the student has a posture change […] When the second determination result is that the student has a posture change, determine multiple frames of classroom images within a preset range centered on the target frame classroom image where the posture change occurs as the target image […] if the second determination result is that the student has a posture change, it is determined that the target frame of the classroom image with the posture change as the center and multiple frames of classroom images within a preset range are used as the target image”), where Fang teaches generate the playback control command for reducing a playback speed of the first image (Paragraph [0026]: “the predefined difference condition may be set as ten percentages”; and Paragraph [0027]: “if the position difference does not conform to the predefined difference condition, the cropping area Ac generated by the auto framing method may be slightly changed relative to the current focus area, so step S100 can be executed for a next stage of detection. If the position difference conforms to the predefined difference condition, step S134 can be executed to set a preset step count and a preset frame interval […] The preset frame interval can be interpreted as time difference between one analyzed frame and another analyzed frame, and can be used to adjust a playback speed of the auto framing animation; an animation speed of the cropping area can be interpreted as a movement speed from the initial detection to the stationary location of the object, so that the preset frame interval can be optionally set in accordance with the animation speed of the cropping area”); and
where Zhang (CN, “697”) teaches in response to the area change being less than or equal to a second threshold (Paragraph [0077]: “The calculated ratio is compared with a preset threshold, and if the ratio is greater than the preset threshold, it is determined that the second determination result is that the student has a posture change”), where Fang teaches generate the playback control command for increasing the playback speed of the first image (Paragraph [0026]: “the predefined difference condition may be set as ten percentages”; and Paragraph [0027]: “if the position difference does not conform to the predefined difference condition, the cropping area Ac generated by the auto framing method may be slightly changed relative to the current focus area, so step S100 can be executed for a next stage of detection. If the position difference conforms to the predefined difference condition, step S134 can be executed to set a preset step count and a preset frame interval […] The preset frame interval can be interpreted as time difference between one analyzed frame and another analyzed frame, and can be used to adjust a playback speed of the auto framing animation; an animation speed of the cropping area can be interpreted as a movement speed from the initial detection to the stationary location of the object, so that the preset frame interval can be optionally set in accordance with the animation speed of the cropping area”).
Regarding claim(s) 18, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 1, where Fang teaches wherein the processor (Figure 1; and Paragraph [0017]) is further configured to:
detect the first image to obtain a center point and a height of the first target object (Paragraph [0020]: “step S100 can be executed to acquire the detection image I1 for object identification; the object identification can be human face detection and/or human body detection”; and Paragraph [0019]: “adjust the visual field to move a middle point or a specific computed point of the foresaid two identification features on a frame center of the screen 14 […] define the final visual field by setting the middle point or the specific computed point of the foresaid two identification features on the frame center of the screen 14”); and
determine a region of interest of the first image according to the center point and the height (Figure 4A-4B; and Paragraph [0025]: “[…] If the number of the identification feature is single, step S124 and step S126 can be executed to draw the marking frame F by containing the identification feature, and extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and the cropping area Ac can be scaled up in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode. If the number of the identification feature is plural, step S128 and step S130 can be executed to find out one or some nearest objects within the visual field for drawing the marking frame F, and then extend the marking frame F in accordance with the preset aspect ratio of the screen 14 for defining the cropping area Ac, and scale up the cropping area Ac in accordance with the size of the screen 14 to be displayed on the screen 14 in the full screen mode”).
Claim(s) 9-12 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (CN 117520577 A; See translation provided by Examiner) in view of Zhang (CN 115830697 A; See translation provided by Examiner) and Fang et al (US 2024/0223881 A1), further in view of Cui et al (US 2024/0314268 A1).
Regarding claim(s) 9, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 1, where Liu teaches wherein the processor is further configured to:
determine (Paragraph [0036]: “obtaining the user's motion information, key points of several movements of the user can be determined from the motion information. These key points are the key connection points when the user moves, such as elbows, wrists, shoulders, waist, hips, knees, ankles”); and
where Zhang (CN, “697”) teaches calculate the first score according to the first displacement(Paragraph [0075]: “Compute the displacement vectors of keypoints in the first and last frames of the continuous multi-frame classroom image”; and Paragraph [0076]: “Calculate the ratio of the mean value of the modulus of each displacement vector to the width of the detection frame. Specifically, the mean value of the modes of the above six displacement vectors is calculated, and then the ratio of the above mean value to the width of the detection frame is calculated”).
Liu, Zhang (CN, “697”) and Fang fails to teach obtain a first audio file corresponding to the first image through the transceiver; and determine a first correlation between the first audio fileCui teaches obtain a first audio file corresponding to the first image through the transceiver (Paragraph [0184]: “S1502: In response to the operation in which the user starts to record the video, an electronic device collects image frames based on a first frame rate by using a photographing apparatus, and collects audio frames by using a recording apparatus”); and
determine a first correlation (read as “audio and motion analysis”) between the first audio file and the first joint (Figure 6; Figure 11; Paragraph [0184]; Paragraph [0145]: “After the scene detection module transmits a motion interval result to the motion analysis module, the motion analysis module marks a motion start tag and a motion end tag based on the motion interval result […]”; and Paragraph [0210]: “the speech recognition module of the video pre-processing algorithm unit performs speech recognition on the audio stream, obtains a speech in the audio stream, and determines audio frames including the speech […]”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Liu, Zhang (CN, “697”), Fang and Cui before the effective filing date of the claimed invention. It would have been obvious to calculate a score based on both displacement of a joint and a correlation between an audio signal and the joint, as Liu teaches evaluating motion based on joint displacement to generate a score, and Cui teaches analyzing audio (speech) and motion information together. The motivation for this combination of references would have been to incorporate temporally corresponding audio information into the displacement-based motion analysis of detected body key points, as Cui teaches obtaining corresponding image and audio frames and correlating the image frames and audio frames on a timeline, thereby providing additional contextual information for evaluating motion and improving the reliability of the resulting evaluation. This motivation for the combination of Liu, Zhang (CN, “697”), Fang and Cui is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 10, Liu as modified by Zhang (CN, “697”), Fang and Cui teaches the electronic device according to claim 9, where Cui teaches wherein the processor (Paragraph [0101]) is further configured to:
perform a speech-to-text conversion on the first audio file to generate text (Paragraph [0173]: “recognizes a speech in the audio stream corresponding to the motion interval by using a speech recognition algorithm. If recognition succeeds, corresponding text is generated”; and Paragraph [0210]: “the speech recognition module of the video pre-processing algorithm unit performs speech recognition on the audio stream, obtains a speech in the audio stream, and determines audio frames including the speech […]”);
determine a number of words in the text (Cui, Paragraph [0177]: “each word in the speech is displayed in image frames corresponding to the word”) associated with the first joint (Liu, Paragraph [0036]); and
determine the first correlation according to the number of the words (where Liu teaches in Paragraph [0035]; Paragraph [0036]; and Paragraph [0042]; and where Cui teaches in Figure 6; Figure 11; Paragraph [0184]; Paragraph [0145]; Paragraph [0177]; and Paragraph [0210]).
Regarding claim(s) 11, Liu as modified by Zhang (CN, “697”), Fang and Cui teaches the electronic device according to claim 9, where Cui teaches wherein the processor (Paragraph [0101]) is further configured to:
determine a first weight of the first displacement and a second weight of the first correlation according to a category of the first image (Figure 6; Figure 11; Paragraph [0184]; Paragraph [0145]; Paragraph [0210]; and Paragraph [0142]: “A scene detection module of the video pre-processing algorithm unit detects a specified action based on the low-resolution preview stream […]”); and
calculate the first score according to the first displacement, the first weight, the first correlation, and the second weight (where Zhang teaches in Paragraph [0075]; and Paragraph [0076]; and where Cui teaches in Figure 6; Figure 11; Paragraph [0184]; Paragraph [0145]; and Paragraph [0210]).
Regarding claim(s) 12, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 1, where Liu teaches wherein the processor (Figure 9; and Paragraph [0092]) is further configured to:
obtain a second image comprising a second target object according to the transceiver (Figure 11; and Paragraph [0025]: “This information can be acquired through an image acquisition device and included in the image information acquired by the device”); and
generate a playback control command for the first image according to the second image (Paragraph [0003]: “The main purpose of this application is to provide a motion evaluation method […], which can solve the technical problem that users cannot understand whether their movements meet the standards and make corresponding adjustments during exercise”; and Paragraph [0036]: “obtaining the user's motion information, key points of several movements of the user can be determined from the motion information. These key points are the key connection points when the user moves, such as elbows, wrists, shoulders, waist, hips, knees, ankles”).
Liu, Zhang (CN, “697”), and Fang fails to teach the output the playback control command through the transceiver. However, Cui teaches the output the playback control command through the transceiver (Figure 6; Figure 11; Paragraph [0009]: “automatically performs processing (for example, frame interpolation or frame extraction) on image frames in the original video file that include the specified action […] slow-motion playing or fast-motion playing is automatically performed on image frames in a video that include a specified action”; Paragraph [0010]: “another electronic device receives the second video file, and plays the second video file, to implement slow-motion playing or fast-motion playing in a motion interval”; Paragraph [0184]; Paragraph [0142]; Paragraph [0145]: “After the scene detection module transmits a motion interval result to the motion analysis module, the motion analysis module marks a motion start tag and a motion end tag based on the motion interval result […]”; and Paragraph [0210]”).
It would have been obvious to generate a playback control command for a first image based on analysis of a second image, as Cui teaches analyzing video frames, motion, and audio information to determine content characteristics, and Liu teaches generating control actions based on analyzed motion. Applying analysis of one image stream to control playback behavior of another image stream represents a predictable use of known multimedia processing techniques. Outputting the control command through a transceiver is a routine implementation. This motivation for the combination of Liu, Zhang (CN, “697”), Fang and Cui is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 19, Liu as modified by Zhang (CN, “697”) and Fang teaches the electronic device according to claim 1, but does not specifically teach wherein the processor is further configured to: output a script through the transceiver, wherein the script comprises the adjustment command and a timestamp of the first image corresponding to the adjustment command.
However, Cui teaches an output a script through the transceiver, wherein the script comprises the adjustment command and a timestamp of the first image corresponding to the adjustment command (Figure 6; Figure 11; Paragraph [0157]: “a video container corresponding to the video 1 includes a first video file and a mark file […] The media framework obtains a motion start tag and a motion end tag in the mark file, and obtains image frames corresponding to a motion interval based on the first video file, the motion start tag, and the motion end tag”; Paragraph [0158]: “The motion start tag corresponds to the twenty-fourth image frame, and the motion end tag corresponds to the twenty-ninth image frame”; and Paragraph [0180]: “the mobile phone determines a motion start tag based on a location of the “Slow motion start” icon 708, determines a motion end tag based on a location of the “Slow motion end” icon 709, and updates a motion start tag and a motion end tag that are stored in a mark file […] performs, based on the updated motion start tag and motion end tag, frame interpolation processing on the image frames in the first video stream that correspond to the motion interval”).
Therefore, It would have been obvious to output a script including an adjustment command and a corresponding timestamp, as generating control commands is taught by Liu, and recording or transmitting such commands along with associated timing information is a well-known and routine practice in multimedia systems. Including a timestamp represents a predictable use of metadata for synchronization and logging purposes. This motivation for the combination of Liu, Zhang (CN, “697”), Fang and Cui is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (CN 117520577 A; See translation provided by Examiner) in view of Zhang (CN 115830697 A; See translation provided by Examiner), Fang et al (US 2024/0223881 A1), and Cui et al (US 2024/0314268 A1), further in view of Zhang et al (CN 110045823 A; See translation provided by Examiner).
Regarding claim(s) 13, Liu as modified by Zhang (CN, “697”), Fang and Cui teaches the electronic device according to claim 12, where Liu teaches wherein the processor (Figure 9; and Paragraph [0092]) is further configured to:
(Paragraph [0003]: “The main purpose of this application is to provide a motion evaluation method […], which can solve the technical problem that users cannot understand whether their movements meet the standards and make corresponding adjustments during exercise”; and Paragraph [0036]: “obtaining the user's motion information, key points of several movements of the user can be determined from the motion information. These key points are the key connection points when the user moves, such as elbows, wrists, shoulders, waist, hips, knees, ankles”); and
in response to the angle being outside the default range (Paragraph [0003]: “The main purpose of this application is to provide a motion evaluation method […], which can solve the technical problem that users cannot understand whether their movements meet the standards and make corresponding adjustments during exercise”), generate the playback control command for pausing the first image (Figure 6; Figure 11; Paragraph [0184]; Paragraph [0145]: “After the scene detection module transmits a motion interval result to the motion analysis module, the motion analysis module marks a motion start tag and a motion end tag based on the motion interval result […]”; Paragraph [0142]: “A scene detection module of the video pre-processing algorithm unit detects a specified action based on the low-resolution preview stream […]”); and Paragraph [0210]: “the speech recognition module of the video pre-processing algorithm unit performs speech recognition on the audio stream, obtains a speech in the audio stream, and determines audio frames including the speech […]”).
Liu, Zhang (CN, “697”), Fang and Cui fails to teach detect the second image to obtain a plurality of limb joints of the second target object, wherein the limb joints comprise a left elbow joint, a right elbow joint, a left knee joint, and a right knee joint; and calculate an angle according to the limb joints.
However, Zhang (CN, “823”) teaches detect the second image to obtain a plurality of limb joints of the second target object, wherein the limb joints comprise a left elbow joint, a right elbow joint, a left knee joint, and a right knee joint (Paragraph [0022]: “key points of the human body in the standard action image […] to identify each joint point, and output the digital signal of the two-dimensional coordinates of each joint point […]); and
calculate an angle according to the limb joints (Paragraph [0063]: “Calculate the angles between the Iimbs according to the two-dimensional coordinates of the key points of the human body in the first skeleton model and the second skeleton model”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the electronic device of Liu as modified by Zhang (CN, “697”), Fang, and Cui with the teachings of Zhang (CN, “823”) to detect a plurality of limb joints of a target object, calculate an angle according to the limb joints, and determine whether the angle is within a predetermined range. The motivation for this combination of references would have been to accurately determine whether a user's movement conforms to a standard movement based on the angles between the user's limbs, as Zhang (CN, “823”) teaches identifying human-body joint points, calculating angles between limbs based on the two-dimensional coordinates of the human-body key points, comparing the angles between the user's movement and the standard movement, and determining whether the user's movement is within an error range, thereby allowing a deviation exceeding a threshold to be identified and corrected. This motivation for the combination of Liu, Zhang (CN, “697”), Fang, Cui, and Zhang (CN, “823”) is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Relevant Prior Art Directed to State of Art
Liu et al (US 2023/0401897 A1) are relevant prior art not applied in the rejection(s) above. Liu discloses an electronic device, comprising: one or more processors; and one or more memories coupled to the one or more processors and configured to store computer program code, the computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform: determining an object user; obtaining a first plurality of image sequences comprising the object user; determining a state of a hand gesture recognition function as a disabled state when determining, based on the first plurality of image sequences, that the object user satisfies a first condition.
Makino (US 2019/0370977 A1) are relevant prior art not applied in the rejection(s) above. Makino discloses a moving object detection apparatus comprising: at least one memory that stores instructions; and at least one processor configured to execute the instructions to: receive video taken by an image capturing apparatus provided in a mobile body; calculate first motions of first pixels of an image included in the video; calculate, based on the first motions of the first pixels, background motions indicating motions of a background area of the image; calculate a transform that generates the background motion; calculate an inverse transform of the transform; apply the inverse transform on the first transform, thereby generating second motions of the first pixels; generate, based on the second motions of the first pixels, third motions of second pixels other than the first pixels; calculate first scores of the first pixels and the second pixels based on the second motions of the first pixels and the third motions of the second pixels; update, based on the transform, background models each representing background pixel values in partial areas of the image; calculate second scores of the first pixels and the second pixels based on the image and the background models of the partial areas, each of the partial areas including a pixel of the first pixels and the second pixels; and detect a moving object from the image based on the first scores and the second scores.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONGBONG NAH whose telephone number is (571)272-1361. The examiner can normally be reached M - F: 7:30am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Edward Urban can be reached on 571-272-7899. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONGBONG NAH/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights