Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1, 14, and 18 are objected to because of the following informalities.
• Claim 1, line 7, “the area of interest”, should read as “the area of user interest”.
• Claim 14, line 3, “a database”, should read as “the database”.
• Claim 18, line 2, “a computer game”, should read as “the computer game”.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 5-13, and 16-20 are rejected under 35 U.S.C. 102 as being unpatentable under US 20240036640 A1 (“Zhang”).
In regards to claim 1, Zhang discloses
An apparatus, comprising ([0028], “The present disclosure describes … apparatuses”): at least one processor system configured to: receive image frames from an image sensor ([0023], “settings … can be applied to a processor … for processing … image frames captured by the image sensor”);
identify, from the image frames, an area of user interest ([0028], “By comparing areas (e.g., in an image) of the XR ([0016], “extended reality (XR)”) environment for which the eyes of the user are focused to areas of the XR environment that are determined to be of potential interest to the user (e.g., the ROIs ([0073], “regions of interest (ROI)”)”);
process, at a first rate ([0063], “a depth sensor … may operate at a different … frame rate from … image sensors”), a first region of the image frames, the first region showing the area of user interest ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”), the first rate being faster than a second rate at which a second region of the image frames is processed ([0064], “image sensors … can use measurements obtained by the accelerometer” Examiner notes that accelerometer measurements can be used to calculate image frame rates.);
and use the processing of the first region at the first rate to render, as part of a computer game ([0076], “a game developer may … be able to designate … an ROI during development of the game”), an extended reality (XR) object as being located at the area of interest ([0028], “the XR environment adds objects … these objects may be considered areas of interest”).
In regards to claim 5, Zhang discloses
wherein the second image frames are different from the first image frames, and wherein the second image frames are received from ([0028], “By comparing areas (e.g., in an image) of the XR environment for which the eyes of the user are focused to areas of the XR environment that are determined to be of potential interest to the user”) the image sensor prior to receipt of the first image frames ([0023], “settings … can be applied to a processor … for processing … image frames captured by the image sensor”).
In regards to claim 6, Zhang discloses
wherein the at least one processor system is configured to ([0023], “settings … can be applied to a processor”): identify the area of user interest based on a determination that ([0028], “By comparing areas (e.g., in an image) of the XR environment for which the eyes of the user are focused to areas of the XR environment that are determined to be of potential interest to the user”) user gaze location corresponds to the area of user interest ([0085], “the eye tracking information indicates a location at which the user's eye is gazing”).
In regards to claim 7, Zhang discloses
wherein the at least one processor system is configured to ([0023], “settings … can be applied to a processor”): identify the area of user interest based on a determination that ([0028], “By comparing areas (e.g., in an image) of the XR environment for which the eyes of the user are focused to areas of the XR environment that are determined to be of potential interest to the user”) user hand location corresponds to the area of user interest ([0024], “an XR system can track parts of the user (e.g., a hand … of a user) to allow the user to interact with items of virtual content.”).
In regards to claim 8, Zhang discloses
wherein the at least one processor system is configured to ([0023], “settings … can be applied to a processor”): identify the area of user interest based on a determination that ([0028], “By comparing areas (e.g., in an image) of the XR environment for which the eyes of the user are focused to areas of the XR environment that are determined to be of potential interest to the user”) user speech corresponds to the area of user interest ([0024], “the XR system … can … include … a speech recognition sensor”).
In regards to claim 9, Zhang discloses
comprising a headset, wherein the headset comprises ([0024], “devices include head-mounted displays”) the image sensor ([0065], “output … can be used by the XR engine … to determine … the head pose … of … image sensors”) and a transparent display ([0025], “an XR system can include an optical ‘see-through’ or ‘pass-through’ display”), and wherein the at least one processor system is configured to ([0023], “settings … can be applied to a processor”): render, on the transparent display, the XR object as part of the computer game ([0054], “the XR system … can perform … rendering of virtual content on a display”).
In regards to claim 10, Zhang discloses
wherein processing the first region at ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”) the first rate comprises executing ([0063], “a depth sensor … may operate at a different … frame rate from … image sensors”) feature extraction at a first frame rate using the image frames from the image sensor ([0069], “compute components … can extract feature points from … images”).
In regards to claim 11, Zhang discloses
wherein the at least one processor system is configured to ([0023], “settings … can be applied to a processor”): process the second region at ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”) the second rate ([0063], “a depth sensor … may operate at a different … frame rate”).
In regards to claim 12, Zhang discloses
wherein the at least one processor system is configured to ([0023], “settings … can be applied to a processor”): process a third region of the image frames at ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”) a third rate ([0063], “a depth sensor … may operate at a different … frame rate”), the third rate being slower than the first rate but faster than the second rate ([0064], “image sensors … can use measurements obtained by the accelerometer” Examiner notes that accelerometer measurements can be used to calculate image frame rates.).
In regards to claim 13, Zhang discloses
wherein the at least one processor system is configured to ([0023], “settings … can be applied to a processor”): process the third region at ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”) the third rate based on ([0063], “a depth sensor … may operate at a different … frame rate”) one or more factors ([0064], “image sensors … can use measurements obtained by the accelerometer” Examiner notes that accelerometer measurements can be used to calculate image frame rates.).
In regards to claim 16, Zhang discloses
A method, comprising: receiving image frames from an image sensor on ([0023], “settings … can be applied to a processor … for processing … image frames captured by the image sensor”) a headset ([0024], “devices include head-mounted displays”);
identifying, from the image frames, an area of user interest ([0028], “By comparing areas (e.g., in an image) of the XR environment for which the eyes of the user are focused to areas of the XR environment that are determined to be of potential interest to the user”);
processing, at a first rate ([0063], “a depth sensor … may operate at a different … frame rate from … image sensors”), a first region of the image frames, the first region showing the area of user interest ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”);
concurrent with processing the first region at the first rate, processing a second region of the image frames at a second rate that is slower than the first rate ([0064], “image sensors … can use measurements obtained by the accelerometer” Examiner notes that accelerometer measurements can be used to calculate image frame rates.);
and using the processing of the first region at the first rate to render a computer-generated object ([0054], “the XR system … can perform … rendering of virtual content on a display”) as appearing in three-dimensional (3D) space in relation to the area of user interest ([0054], “the XR system … can generate a … a three-dimensional (3D) map) of an environment”).
In regards to claim 17, Zhang discloses
comprising: using the processing of the second region at ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”) the second rate to monitor ([0063], “a depth sensor … may operate at a different … frame rate”) the real-world environment of a user while the user plays a computer game ([0054], “The display … allows a user to see the real-world environment and also allows XR content to be … displayed thereon”).
In regards to claim 18, Zhang discloses
wherein the computer-generated object comprises an object being rendered as part of ([0054], “the XR system … can perform … rendering of virtual content on a display”) a computer game ([0076], “a game developer may … be able to designate … an ROI during development of the game”).
In regards to claim 19, Zhang discloses
An apparatus, comprising: at least one computer readable storage medium (CRSM) that is not a transitory signal, the at least one CRSM comprising ([0102], “Storage device … can be a … non-transitory and … computer-readable memory device”) instructions executable by a processor system to ([0100], “Processor … can include … a … processor where … instructions are incorporated into the … processor design”): receive image frames from an image sensor ([0023], “settings … can be applied to a processor … for processing … image frames captured by the image sensor”);
identify, from the image frames, an area of user interest ([0028], “By comparing areas (e.g., in an image) of the XR environment for which the eyes of the user are focused to areas of the XR environment that are determined to be of potential interest to the user (e.g., the ROIs”);
based on the identification of the area of user interest, process, at a first rate ([0063], “a depth sensor … may operate at a different … frame rate from … image sensors”), a first region of the image frames, the first region showing the area of user interest ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”);
and use the processing of the first region at the first rate to render a virtual object in relation to the area of user interest ([0054], “the XR system … can perform … rendering of virtual content on a display”).
In regards to claim 20, Zhang discloses
wherein the instructions are executable to: concurrent with processing the first region at the first rate, process a second region of the image frames at ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”) a second rate to monitor ([0063], “a depth sensor … may operate at a different … frame rate”) the real-world environment of a user ([0054], “The display … allows a user to see the real-world environment and also allows XR content to be … displayed thereon”), the second rate being slower than the first rate ([0064], “image sensors … can use measurements obtained by the accelerometer” Examiner notes that accelerometer measurements can be used to calculate image frame rates.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-4 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable under Zhang in view of US 20230221833 A1 (“Holder”).
In regards to claim 2, Zhang discloses the following limitations with the exception of the underlined limitation.
wherein the at least one processor system is configured to: access ([0023], “settings … can be applied to a processor) a database to identify that the first region shows the area of user interest, the database indicating different potential items of user interest ([0073], “data indicating which area(s) of an image at which a user is gazing may be compared to regions of interest”).
Holder discloses
a database to identify that ([0087], “the database … is … stored in a memory”)
Zhang and Holder are considered analogous to the claimed invention because they are in the fields of reality systems and reality experiences. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the applicant’s invention for an apparatus, comprising: at least one processor system configured to: receive image frames from an image sensor; identify, from the image frames, an area of user interest; process, at a first rate, a first region of the image frames, the first region showing the area of user interest, the first rate being faster than a second rate at which a second region of the image frames is processed; and use the processing of the first region at the first rate to render, as part of a computer game, an extended reality (XR) object as being located at the area of interest, wherein the at least one processor system is configured to: access, the first region shows the area of user interest, the database indicating different potential items of user interest, as disclosed by Zhang, a database to identify that, as disclosed by Holder, to provide a database for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display. One skilled in the art would understand and recognize the value of the addition of a database for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display.
In regards to claim 3, Zhang discloses the following limitations with the exception of the underlined limitation.
wherein the at least one processor system is configured to: identify the first region based on accessing ([0023], “settings … can be applied to a processor) the database and based on user gaze location ([0085], “the eye tracking information indicates a location at which the user's eye is gazing”).
Holder discloses
the database ([0087], “the database … is … stored in a memory”)
Zhang and Holder are considered analogous to the claimed invention because they are in the fields of reality systems and reality experiences. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the applicant’s invention for an apparatus, comprising: at least one processor system configured to: receive image frames from an image sensor; identify, from the image frames, an area of user interest; process, at a first rate, a first region of the image frames, the first region showing the area of user interest, the first rate being faster than a second rate at which a second region of the image frames is processed; and use the processing of the first region at the first rate to render, as part of a computer game, an extended reality (XR) object as being located at the area of interest, wherein the at least one processor system is configured to: access, the first region shows the area of user interest, the database indicating different potential items of user interest, and based on user gaze location, as disclosed by Zhang, the database, as disclosed by Holder, to provide a database for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display. One skilled in the art would understand and recognize the value of the addition of a database for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display.
In regards to claim 4, Zhang discloses the following limitations with the exception of the underlined limitation.
wherein the image frames are first image frames, and wherein the at least one processor system is configured to: based on second image frames from the image sensor ([0023], “settings … can be applied to a processor … for processing … image frames captured by the image sensor) configure the database with the different potential items of user interest ([0075], “ROIs … are portions of the content … visible to a user … which are identified as … of interest to the user”).
Holder discloses
configure the database with ([0087], “the database … is … stored in a memory”)
Zhang and Holder are considered analogous to the claimed invention because they are in the fields of reality systems and reality experiences. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the applicant’s invention for an apparatus, comprising: at least one processor system configured to: receive image frames from an image sensor; identify, from the image frames, an area of user interest; process, at a first rate, a first region of the image frames, the first region showing the area of user interest, the first rate being faster than a second rate at which a second region of the image frames is processed; and use the processing of the first region at the first rate to render, as part of a computer game, an extended reality (XR) object as being located at the area of interest, wherein the at least one processor system is configured to: access, the first region shows the area of user interest, the database indicating different potential items of user interest, wherein the image frames are first image frames, and wherein the at least one processor system is configured to: based on second image frames from the image sensor, the different potential items of user interest, as disclosed by Zhang, a database to identify that and configure the database with, as disclosed by Holder, to provide a database for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display. One skilled in the art would understand and recognize the value of the addition of a database for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display.
In regards to claim 14, Zhang discloses the following limitations with the exception of the underlined limitation.
wherein the one or more factors comprise one or more of: previous user interaction with an object shown in the third region ([0024], “an XR system can track parts of the user … to allow the user to interact with items of virtual content”), the object being indicated in a database as an interactive object for gameplay ([0076], “a game developer may … be able to designate … an ROI during development of the game”).
Holder discloses
the object being indicated in a database as an interactive object for ([0087], “the database … is … stored in a memory”)
Zhang and Holder are considered analogous to the claimed invention because they are in the fields of reality systems and reality experiences. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the applicant’s invention for an apparatus, comprising: at least one processor system configured to: receive image frames from an image sensor; identify, from the image frames, an area of user interest; process, at a first rate, a first region of the image frames, the first region showing the area of user interest, the first rate being faster than a second rate at which a second region of the image frames is processed; and use the processing of the first region at the first rate to render, as part of a computer game, an extended reality (XR) object as being located at the area of interest, wherein the at least one processor system is configured to: process the second region at the second rate, wherein the at least one processor system is configured to: process a third region of the image frames at a third rate, the third rate being slower than the first rate but faster than the second rate, wherein the at least one processor system is configured to: process the third region at the third rate based on one or more factors, wherein the one or more factors comprise one or more of: previous user interaction with an object shown in the third region, and gameplay, as disclosed by Zhang, the object being indicated in a database as an interactive object for, as disclosed by Holder, to provide a database for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display. One skilled in the art would understand and recognize the value of the addition of a database for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display.
In regards to claim 15, Zhang discloses the following limitation with the exception of the underlined limitations.
wherein the one or more factors comprise the previous user interaction with the object shown in the third region, and wherein the previous user interaction comprises one or more of ([0024], “an XR system can track parts of the user … to allow the user to interact with items of virtual content”): the user previously looking at the object, the user previously touching the object.
Holder discloses
the user previously looking at the object ([0034], “A person may … interact with a XR object using … sight”), the user previously touching the object ([0034], “A person may … interact with a XR object using … touch”).
Zhang and Holder are considered analogous to the claimed invention because they are in the fields of reality systems and reality experiences. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the applicant’s invention for an apparatus, comprising: at least one processor system configured to: receive image frames from an image sensor; identify, from the image frames, an area of user interest; process, at a first rate, a first region of the image frames, the first region showing the area of user interest, the first rate being faster than a second rate at which a second region of the image frames is processed; and use the processing of the first region at the first rate to render, as part of a computer game, an extended reality (XR) object as being located at the area of interest, wherein the at least one processor system is configured to: process the second region at the second rate, wherein the at least one processor system is configured to: process a third region of the image frames at a third rate, the third rate being slower than the first rate but faster than the second rate, wherein the at least one processor system is configured to: process the third region at the third rate based on one or more factors, wherein the one or more factors comprise one or more of: previous user interaction with an object shown in the third region, and gameplay, wherein the one or more factors comprise the previous user interaction with the object shown in the third region, and wherein the previous user interaction comprises one or more of, as disclosed by Zhang, the object being indicated in a database as an interactive object for, the user previously looking at the object, the user previously touching the object, as disclosed by Holder, to provide a database and a XR object for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display. One skilled in the art would understand and recognize the value of the addition of a database and a XR object for computer systems that provide computer-generated experiences with virtual reality and mixed reality experiences via a display.
Contact Information
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to Lisa Antoine whose telephone number is
(571) 272-4252 and whose email address is lantoine@uspto.gov. The examiner can be reached Monday-Thursday, 7:30 am – 5:30 pm CT. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xuan Thai, can be reached on (571) 272-7147. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Publication Information
Information regarding the status of published or unpublished applications may be
obtained from the Patent Center. Unpublished application information in the Patent
Center is available to registered users. To file and manage patent submissions in the
Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about the
Patent Center and https://www.uspto.gov/patents/docx for information about filing in
DOCX format. For additional questions, contact the Electronic Business Center (EBC)
at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer
Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LISA H ANTOINE/
Examiner, Art Unit 3715
/XUAN M THAI/Supervisory Patent Examiner, Art Unit 3715
Prosecution Insights