DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is made non-final.
Claims 1-20 are pending in the case. Claims 1, 12, and 20 are independent claims.
Claim Objections
Claim 6 is objected to because of the following informalities:
Claim 6 recites “the camera” but, because no camera was previously recited, should recite “a camera”.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 7, 9, 12-14, 18, and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Blackstone et al. (US 2014/0298273 A1).
Regarding claim 1, Blackstone teaches a host (at least image processing system 12 of FIG. 1 corresponding to image processing system 20 of FIG. 2, [0066-0074]), comprising:
a storage circuit, configured to store a program code (memory 28 of FIG. 2 and [0070]); and
a processor, coupled to the storage circuit and configured to access the program code (processor 22 of FIG. 2 and [0070-0071]) to execute:
obtaining an environment image of an environment around a user (camera interface 24 of FIG. 2 and [0070-0072]: Environment image of an environment around a user is obtained, allowing identification of the user’s hand gestures; step 254 of FIG. 10 and [0101]);
identifying one or more objects in the environment based on the environment image (FIG. 2 and [0070-0072]: As supported in [0071], “the interactive application 34 and the operating system 30 configure the processor 22 to generate and render an initial user interface using a set of interface objects. The interface objects can be modified in response to a detected interaction with a targeted interface object and an updated user interface rendered. Targeting and interaction with interface objects can be performed via a 3D gesture based input modality using the 3D gesture tracking application 32.” Thus, based on the environment image, one or more objects/interface objects are identified, the one or more objects of which are targetable via a 3D gesture; step 256 of FIG. 10 and [0101]; FIG. 11A and [0105]: for example, an environment image may include the user interface 270 seen in FIG. 11A, and the “user interface 270 is displayed including a plurality of interface objects, one of which includes a tile graphical element 272 enabling launching of a media player application”);
performing a hand tracking to determine a hand track of a hand of the user (3D gesture tracking application 32 of FIG. 2 and [0070-0072]: As supported in [0071], “the 3D gesture tracking application 32 and the operating system 30 configure the processor 22 to capture image data using an image capture system via the camera interface 24, and detect a targeting 3D gesture in the captured image data that identifies a targeted interface object within a user interface”. As further supported in at least [0070], a 3D gesture tracking application identifies 3D gestures such as hand gestures; step 256 of FIG. 10 and [0101]; FIGS. 11A-B and [0105-0106]: hand tracking is performed, tracking the user’s pointing 3D gesture as indicated by cursor 274);
determining one or more pointing periods of the one or more objects based on the hand track (step 256 of FIG. 10 and [0100-0101]: an object is determined to be a target object when a pointing 3D gesture persists for a predetermined period of time; FIG. 2 and [0070-0072]: As supported in [0071], “the 3D gesture tracking application 32 and the operating system 30 configure the processor 22 to capture image data using an image capture system via the camera interface 24, and detect a targeting 3D gesture in the captured image data that identifies a targeted interface object within a user interface”; FIG. 11B and [0105-0106]: As supported in [0105], “The user can target an interface object by moving the cursor into the target zone of the interface object using a pointing 3D gesture and holding the cursor within the target zone for a predetermined period of time”. In this example, a pointing period for tile 272 is determined);
determining one of the one or more objects as a target object based on the one or more pointing periods (step 256 of FIG. 10 and [0100-0101]: an object is determined to be a target object when a pointing 3D gesture persists for a predetermined period of time; FIG. 11B and [0105-0106]: As supported in [0105], “The user can target an interface object by moving the cursor into the target zone of the interface object using a pointing 3D gesture and holding the cursor within the target zone for a predetermined period of time”. In this example, tile 272 is determined to be a target object because the pointing period for tile 272 reaches or exceeds a predetermined period of time); and
performing an object-based operation based on the target object (steps 264 of FIG. 10 and [0100-0103]: an object-based operation based on the target object is performed; FIGS. 11B-C and [0106-0107]: media player application is launched based on the selection of the tile 272/target object).
Regarding claim 2, Blackstone further teaches the host according to claim 1, wherein the processor is further configured to access the program code to execute:
determining a longest pointing period out of the one or more pointing periods (step 256 of FIG. 10 and [0100-0101]: an object is determined to be a target object when a pointing 3D gesture persists for a predetermined period of time; FIG. 2 and [0070-0072]: As supported in [0071], “the 3D gesture tracking application 32 and the operating system 30 configure the processor 22 to capture image data using an image capture system via the camera interface 24, and detect a targeting 3D gesture in the captured image data that identifies a targeted interface object within a user interface”; FIG. 11B and [0105-0106]: As supported in [0105], “The user can target an interface object by moving the cursor into the target zone of the interface object using a pointing 3D gesture and holding the cursor within the target zone for a predetermined period of time”. In this example, a pointing period for tile 272 is determined to be the longest as other objects have a pointing period of zero);
determining an object of the one or more objects corresponding to the longest pointing period as the target object (step 256 of FIG. 10 and [0100-0101]: an object is determined to be a target object when a pointing 3D gesture persists for a predetermined period of time; FIG. 11B and [0105-0106]: As supported in [0105], “The user can target an interface object by moving the cursor into the target zone of the interface object using a pointing 3D gesture and holding the cursor within the target zone for a predetermined period of time”. In this example, tile 272 is determined to be a target object because the pointing period for tile 272 reaches or exceeds a predetermined period of time and is also the longest as other objects have a pointing period of zero).
Regarding claim 3, Blackstone further teaches the host according to claim 1, the processor is further configured to access the program code to execute:
determining that whether a pointing period of the one or more pointing periods is greater than a predetermined threshold period; and
in response to the pointing period being greater than the predetermined threshold period, determining an object of the one or more objects corresponding to the pointing period as the target object (step 256 of FIG. 10 and [0100-0101]: an object is determined to be a target object when a pointing 3D gesture persists for a predetermined period of time; FIG. 11B and [0105-0106]: As supported in [0105], “The user can target an interface object by moving the cursor into the target zone of the interface object using a pointing 3D gesture and holding the cursor within the target zone for a predetermined period of time”. In this example, tile 272 is determined to be a target object because the pointing period for tile 272 reaches or exceeds a predetermined period of time).
Regarding claim 7, Blackstone further teaches the host according to claim 1, wherein the processor is further configured to access the program code to execute:
obtaining a hand tracking video of the hand tracking (camera interface 24 of FIG. 2 and [0072]);
obtaining a target frame of the hand tracking video as a target image based on the pointing period (step 256 of FIG. 10 and [0100-0101]: an object is determined to be a target object when a pointing 3D gesture persists for a predetermined period of time; FIG. 11B and [0105-0106]: As supported in [0105], “The user can target an interface object by moving the cursor into the target zone of the interface object using a pointing 3D gesture and holding the cursor within the target zone for a predetermined period of time”. In this example, tile 272 is determined to be a target object because the pointing period for tile 272 reaches or exceeds a predetermined period of time; camera interface 24 of FIG. 2 and [0072]: while frames of a video are received by the processor via camera interface 24, a target frame, and likewise target image, may be a frame in which the pointing 3D gesture is held within the target zone for tile 272 during the predetermined period of time); and
performing the object-based operation based on the target image (camera interface 24 of FIG. 2 and [0072], steps 264 of FIG. 10 and [0100-0103]: based on the target image which helps identify a target object, an object-based operation based on the target object is performed; FIGS. 11B-C and [0106-0107]: media player application is launched based on the selection of the tile 272, which was determined to be the target object based on the target image).
Regarding claim 9, Blackstone further teaches the host according to claim 7, wherein the processor is further configured to access the program code to execute:
cropping the target object of the target image as a region of interest; and
performing the object-based operation based on the region of interest (steps 256 and 264 of FIG. 10 and [0100-0104]: An object is determined to be a target object when a pointing 3D gesture persists for a predetermined period of time. An object-based operation based on the target object is performed; FIGS. 11B-C and [0105-0107]: As supported in [0105], “The user can target an interface object by moving the cursor into the target zone of the interface object using a pointing 3D gesture and holding the cursor within the target zone for a predetermined period of time”. In this example, tile 272 is determined to be a target object because the pointing period for tile 272 reaches or exceeds a predetermined period of time. The target object/tile 272 is cropped as a region of interest as opposed to other portions of the user interface 270. The media player application is launched based on the selection of the tile 272 and the region of interest).
Regarding claims 12-14 and 18, the claims recite an object-based operation system, comprising: a camera, configured to obtain an environment image of an environment around a user; a display configured to display information about the environment to the user; a storage circuit, configured to store a program code; and a processor, coupled to the storage circuit and configured to access the program code (FIG. 1 corresponding to image processing system 20 of FIG. 2, [0066-0074]) to execute operations corresponding to the method of claims 1-3 and 7, respectively, and are therefore rejected on the same premises.
Regarding claim 20, the claim recites an object-based operation method, comprising steps corresponding to the host of claim 1 and is therefore rejected on the same premise.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5, 6, 16, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Blackstone et al. (US 2014/0298273 A1), in view of Roth (US 2026/0086650 A1).
Regarding claim 5, Blackstone teaches the host according to claim 1.
Blackstone does not explicitly teach wherein the object-based operation is an artificial intelligence query.
Roth teaches wherein the object-based operation is an artificial intelligence query (FIG. 7A and [0066]: the object-based operation is an artificial intelligence query).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Blackstone by incorporating the teachings of Roth so as to include wherein the object-based operation is an artificial intelligence query. Doing so would allow the user to query artificial intelligence assistance to help more efficiently solve the user’s inquiries. In this way, the user may save time by not having to manually research for an answer.
Regarding claim 6, Blackstone in view of Roth teaches the host according to claim 5.
Roth further teaches wherein the processor is further configured to access the program code to execute:
in response to receiving query content of the artificial intelligence query from the user, enabling the camera (FIG. 7A and [0066]: in response to receiving query content of the artificial intelligence query from the user, the camera a camera is enabled).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Blackstone in view of Roth by incorporating the further teachings of Roth so as to include in response to receiving query content of the artificial intelligence query from the user, enabling the camera. Doing so would help reduce the amount of inputs the user may need to perform to allow the artificial intelligence to accurately assess the user’s issue. By enabling the camera in response to receiving query content of the artificial intelligence query from the user, the camera can be leveraged by the artificial intelligence to help resolve the user’s issues, rather than the user having to type, verbally explain, or so on, improving convenience for the user.
Regarding claims 16 and 17, the claims recite an object-based operation system corresponding to the method of claims 5 and 6, respectively, and are therefore rejected on the same premises.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Blackstone et al. (US 2014/0298273 A1), in view of Wipfler et al (US 2025/0103687 A1).
Regarding claim 10, Blackstone teaches the host according to claim 7, wherein the processor is further configured to access the program code to execute:
cropping a target area… as a region of interest; and
performing the object-based operation based on the region of interest (steps 256 and 264 of FIG. 10 and [0100-0104]: An object is determined to be a target object when a pointing 3D gesture persists for a predetermined period of time. An object-based operation based on the target object is performed; FIGS. 11B-C and [0105-0107]: As supported in [0105], “The user can target an interface object by moving the cursor into the target zone of the interface object using a pointing 3D gesture and holding the cursor within the target zone for a predetermined period of time”. In this example, tile 272 is determined to be a target object because the pointing period for tile 272 reaches or exceeds a predetermined period of time. The target object/tile 272 is cropped as a region of interest as opposed to other portions of the user interface 270. The media player application is launched based on the selection of the tile 272 and the region of interest).
Blackstone does not explicitly teach cropping a target area extending a specific distance from the target object of the target image.
Wipfler teaches cropping a target area extending a specific distance from the target object of the target image ([0015]: “images cropped within a predetermined distance from the outline of the object”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the cropping a target area of Blackstone by incorporating the teachings of Wipfler so as to include cropping a target area extending a specific distance from the target object of the target image. Doing so would help ensure that the target object is fully captured in the region of interest to allow accurate performance of the relevant object-based operation.
Claim(s) 11 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Blackstone et al. (US 2014/0298273 A1), in view of Lemus et al. (US 2015/0007055 A1).
Regarding claim 11, Blackstone teaches the host according to claim 1.
Although Blackstone teaches determining hand gestures based on the hand tracking (3D gesture tracking application 32 of FIG. 2 and [0070-0072]: As supported in [0071], “the 3D gesture tracking application 32 and the operating system 30 configure the processor 22 to capture image data using an image capture system via the camera interface 24, and detect a targeting 3D gesture in the captured image data that identifies a targeted interface object within a user interface”. As further supported in at least [0070], a 3D gesture tracking application identifies 3D gestures such as hand gestures; step 256 of FIG. 10 and [0101]; FIGS. 11A-B and [0105-0106]: hand tracking is performed, tracking the user’s pointing 3D gesture as indicated by cursor 274), Blackstone does not explicitly teach wherein the processor is further configured to access the program code to execute: determining that whether the hand is in a tagging gesture or not based on the hand tracking; and in response to the hand being in the tagging gesture, assigning a tag to the target object based on the tagging gesture.
Lemus teaches determining that whether the hand is in a tagging gesture or not based on the hand tracking; and
in response to the hand being in the tagging gesture, assigning a tag to the target object based on the tagging gesture (FIGS. 5-8 and [0028-0035]: for example, a user selects object 402-1 by drawing a circle and then draws a checkmark 604 as a tag gesture. The target object/object 402-1 is assigned a tag based on the tagging gesture).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the hand tracking of Blackstone by incorporating the teachings of Lemus so as to include determining that whether the hand is in a tagging gesture or not based on the hand tracking; and in response to the hand being in the tagging gesture, assigning a tag to the target object based on the tagging gesture. Doing so would allow the user to organize objects in a customizable manner for more effective future reference, as opposed to a collection of untagged objects.
Regarding claim 19, the claim recites an object-based operation system corresponding to the host of claim 11 and is therefore rejected on the same premise.
Allowable Subject Matter
Claims 4, 8, and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure, including:
US 11620000 B1: gesture meeting or exceeding a threshold fixation level to control input target
US 2019/0361521 A1: target engagement input in a virtual environment
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNY NGUYEN whose telephone number is (571)272-4980. The examiner can normally be reached M-Th 7AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KIEU D VU can be reached on (571)272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNY NGUYEN/Primary Examiner, Art Unit 2171
Prosecution Insights