Prosecution Insights
Last updated: November 20, 2026
Application No. 19/022,906

VERTEX POSE ADJUSTMENT WITH PASSTHROUGH AND TIME-WARP TRANSFORMATIONS FOR VIDEO SEE-THROUGH (VST) EXTENDED REALITY (XR)

Non-Final OA §103
Filed
Jan 15, 2025
Priority
Jan 19, 2024 — provisional 63/622,874
Examiner
PRINGLE-PARKER, JASON A
Art Unit
2617
Tech Center
2600 — Communications
Assignee
Samsung Electronics Co., Ltd.
OA Round
1 (Non-Final)
84%
Grant Probability
Favorable
1-2
OA Rounds
5m
Est. Remaining
98%
With Interview

Examiner Intelligence

Grants 84% — above average
84%
Career Allowance Rate
480 granted / 573 resolved
+21.8% vs TC avg
Moderate +14% lift
Without
With
+13.7%
Interview Lift
resolved cases with interview
Typical timeline
2y 3m
Avg Prosecution
23 currently pending
Career history
605
Total Applications
across all art units

Statute-Specific Performance

§101
10.7%
-29.3% vs TC avg
§103
49.4%
+9.4% vs TC avg
§102
21.1%
-18.9% vs TC avg
§112
11.4%
-28.6% vs TC avg
Black line = Tech Center average estimate • Based on career data from 573 resolved cases

Office Action

§103
Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . DETAILED ACTION Allowable Subject Matter Claims 4-5, 7, 11-12, 14, 18-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim(s) 1-3, 8-10, 15-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ishihara “Integrating Both Parallax and Latency Compensation into Video See-through Head-mounted Display” in view of Pohl U.S. Patent/PG Publication 20230022934. Regarding claim 1 (independent): A method comprising: determining, using at least one processing device of an extended reality (XR) device, (Ishihara Fig. 1: (a) Our head-mounted display prototype.)(Ishihara 1 There are two types of MR: optical see-through (OST) and video see-through (VST). In OST, users see the real world directly through transparent displays; in contrast, VST captures the world using cameras mounted in front of the head-mounted display (HMD) and projects it onto a display.) a first set of vertex adjustment values of a distortion (Ishihara 4.1 The reprojection operator converts the depth image DA in view at the time A =(PA,VA) to another depth image DB in view at the time B =(PB,VB). It is denoted as DB =BQA(DA) where DA and DB are depth images, and BHA is the reproj). receiving, using the at least one processing device, image frame data of a scene captured at a first time and at a first head pose using a see-through camera of the XR device (Ishihara 4.4 of cameras. A pair of color cameras is used to capture the visible image and a pair of IR cameras with an IR dot projector are applied for depth sensing (referred to as a depth camera) . All cameras start their exposures at the same time, although the exposure duration differs between the color and IR cameras. Raw images taken by the stereo color camera are developed and stereo-rectified. The processed result is denoted as the grabbed color image Cg,i viewed from Ci =(Pc,Vc[tg,i]).)(Ishihara 4.2 The tracker block estimates the pose of device X at the current time t0 and also predicts the pose at a future time t by extrapolating the estimated results until now. These poses are denoted as ˆVX[t0] and ˆ VX(t|t0), respectively. To do this, the tracker block first estimates or predicts the camera pose ˆVsl using inputs from the stereo cameras sl and sr and the IMU, using a visual-inertial odometry (VIO) method similar to that described in [4]. Then, the estimated or predicted camera poses are converted to the pose of device X using the pre-calibrated transform matrix XTsl.). applying, using the at least one processing device, the first set of vertex adjustment values of the distortion (Ishihara 4.1 The reprojection operator converts the depth image DA in view at the time A =(PA,VA) to another depth image DB in view at the time B =(PB,VB). It is denoted as DB =BQA(DA) where DA and DB are depth images, and BHA is the reprojection matrix that maps points in DA to corresponding points in DB.)(Ishihara 4.53 Fourthly, the refined tuple of the foreground depth and color images is reprojected to the view matched with the renderer. This process can be represented as:)(Ishihara 4.6 This block composes the color and depth images of the reconstructed foreground Bfr,i[Rj], the reconstructed background Bbd,i[Rj], and the rendered virtual scenes Bv,j[Rj]. Occlusion between the virtual and reconstructed scenes is realized by depth testing between input images. The resulting composition is denoted as Cm,j[Rj])(Ishihara 4.2 To do this, the tracker block first estimates or predicts the camera pose ˆVsl using inputs from the stereo cameras sl and sr and the IMU, using a visual-inertial odometry (VIO) method similar to that described in [4]. Then, the estimated or predicted camera poses are converted to the pose of device X using the pre-calibrated transform matrix XTsl.). predicting, using the at least one processing device, a second head pose at a second time subsequent to the first time (Ishihara 4.7 The tracker block estimates the pose of device X at the current time t0 and also predicts the pose at a future time t by extrapolating the estimated results until now. These poses are denoted as ˆVX[t0] and ˆ VX(t|t0), respectively. To do this, the tracker block first estimates or predicts the camera pose ˆVsl using inputs from the stereo cameras sl and sr and the IMU, using a visual-inertial odometry (VIO) method similar to that described in [4]. Then, the estimated or predicted camera poses are converted to the pose of device X using the pre-calibrated transform matrix XTsl.). generating, using the at least one processing device, based on the predicted second head pose, a second set of vertex adjustment values of the distortion (Ishihara Fig. 5: Predicted head pose on rendering and on warping.) (Ishihara 4.7 This warper block applies image warping (referred to as “temporal warping”) to the composed scene Cm,j[Rj], to correct the prediction error of the user’s head pose used at rendering Ve(tr,j+1|tr,j) with the latest one predicted just before display scanning Ve(tp,k|tw,k) . And this warper corrects the distortion produced by the display optics. The warped results are sent to the display panel. Temporal warping is a type of inverse warping in which the source image Cm,j[Rj] is assumed to be located on a plane (a “stabilization plane”) but its destination plane is twisted by user head movements as the display scans from top to bottom. This twisted destination mesh is formed by independently reproject ing the upper and lower corners of the composed scene. The upper left and right corners of the composed scene are re-projected to the eye space as viewed from the predicted eye pose at the start of display scanning Ve(tp,k|tw,k) . The lower left and right corners are reprojected to the eye space as viewed from the predicted eye pose at the end of display scanning Ve(tp,k+1|tw,k) , as shown in Figure 8). and displaying the rendered virtual frame by the XR device at the second time, the rendered virtual frame comprising a corrected view of the scene (Ishihara 3.2 The renderer block renders virtual objects (as viewed from the tracked head pose) as virtual scene images. The composite block composes the virtual and reconstructed scenes in the composed image. The warper block applies image warping to the composed im age, to match the head motions and correct the distortion of the display optics. The display block presents a warped image of the user’s eyes, using near-eye display panels and optics.). Ishihara does not teach a distortion mesh. In a related field of endeavor, Pohl teaches: A method comprising: determining, using at least one processing device of an extended reality (XR) device, (Pohl [0012] The illustrated system 100 includes a distortion compensation system 130 in communication with lens 120. In some embodiments, the distortion compensation system 130 receives an input image 103 from an input source such as, for example, the AV source 102 and/or lens 120. The illustrated distortion compensation system 130 includes a system processor 135 and logic module 140 to perform processes to reduce distortions within the received input images. In some embodiments, the distortion compensation system 130 may be completely or partially incorporated within the HMD 110. In at least some embodiments, the logic module 140 includes an image mapping module 150, radial aberration compensation (RAC) module 160, chromatic aberration compensation (CAC) module 170, blending module 180, and memory 190 having a frame buffer 195.) a first set of vertex adjustment values of a distortion mesh (Pohl [0014] In at least one embodiment, the image mapping module 150 receives one or more input images and maps the received images onto one or more of a plurality of distortion meshes.) receiving, using the at least one processing device, image frame data of a scene captured at a first time and at a first head pose using a see-through camera of the XR device (Pohl [0012] The illustrated system 100 includes a distortion compensation system 130 in communication with lens 120. In some embodiments, the distortion compensation system 130 receives an input image 103 from an input source such as, for example, the AV source 102 and/or lens 120.) applying, using the at least one processing device, the first set of vertex adjustment values of the distortion mesh to the image frame data to obtain intermediate image data (Pohl [0013] In at least one embodiment, the distortion compensation system 130 receives input images captured from an image source such as, for example, the AV source 102, and causes the logic module 140 to perform processes to map a received image onto one or more of a plurality of distortion meshes, to compensate for (e.g., correct), radial distortions and chromatic aberrations within the received images, to blend the corrected images, and output a blended/corrected image to the HMD 110 such that a corrected image having reduced distortions can be viewed by a user.) and displaying the rendered virtual frame by the XR device at the second time, the rendered virtual frame comprising a corrected view of the scene (Pohl [0090] The apparatus additionally includes composition circuitry to generate a composite corrected image from the multiple layers of the computer generated image, the composite corrected image blended from the multiple layers of the computer generated image and output circuitry to present the composite corrected image to the electronic display device. In one embodiment, the distortion meshes can be automatically generated.) Therefore, it would have been obvious before the effective filing date of the claimed invention to use a distortion mesh as taught by Pohl. The motivation for doing so would have been that the distortion mesh provides correction, preventing aberrations (Pohl [0013]-[0015]). Further, the rationale for doing so would have been that it combines prior art elements according to known methods to yield predictable results since Ishihara corrects display distortion and Pohl corrects display distortion using a mesh, where the input on both is an HMD image and the output for both is a corrected displayed image on the HMD. Therefore it would have been obvious to combine Pohl with Ishihara to obtain the invention. Regarding claim 2: The method of claim 1, has all of its limitations taught by Ishihara in view of Pohl. Ishihara further teaches wherein: the first set of vertex adjustment values compensates for static differences between the image frame data and the corrected view of the scene and the static differences comprise at least one of: (Ishihara 1 To reconstruct captured images, we reproject images captured by the color camera to the user’s eye position, using depth maps. Reprojected images exhibit disocclusion areas, attributable to positional differences between the cameras and eyes.). Regarding claim 3: The method of claim 1, has all of its limitations taught by Ishihara in view of Pohl. Ishihara further teaches wherein the second head pose comprises a time-warp transformation of the first head pose (Ishihara 1 In the second warp, we use two-dimensional warping to match the composed scenes with the user’s head motions.)(Ishihara 4.7 This warper block applies image warping (referred to as “temporal warping”) to the composed scene Cm,j[Rj], to correct the prediction error of the user’s head pose used at rendering Ve(tr,j+1|tr,j) with the latest one predicted just before display scanning Ve(tp,k|tw,k) . And this warper corrects the distortion produced by the display optics. The warped results are sent to the display panel. Temporal warping is a type of inverse warping in which the source image Cm,j[Rj] is assumed to be located on a plane (a “stabilization plane”) but its destination plane is twisted by user head movements as the display scans from top to bottom.)(Ishihara 2.1 Many forward warping methods have been proposed, including the merging of multi-point inputs as point clouds [37], the genera tion of interpolated perspectives using video and depth inputs [63], hole-filling using multi-perspective inputs [39], foreground silhouette guides [9], geometric defect compensation [14], remote rendering im plementation [58], graphics processing unit (GPU) optimization [57], and real-time processing in stand-alone HMDs [15].). Regarding claim 8 (independent): The claim is a parallel version of claim 1. As such it is rejected under the same teachings. Regarding claim 9: The claim is a parallel version of claim 2. As such it is rejected under the same teachings. Regarding claim 10: The claim is a parallel version of claim 3. As such it is rejected under the same teachings. Regarding claim 15 (independent): The claim is a parallel version of claim 1. As such it is rejected under the same teachings. Regarding claim 16: The claim is a parallel version of claim 2. As such it is rejected under the same teachings. Regarding claim 17: The claim is a parallel version of claim 3. As such it is rejected under the same teachings. Claim(s) 6, 13, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ishihara “Integrating Both Parallax and Latency Compensation into Video See-through Head-mounted Display” in view of Pohl U.S. Patent/PG Publication 20230022934. Regarding claim 6: The method of claim 1, has all of its limitations taught by Ishihara in view of Pohl. Pohl further teaches wherein applying the first set of vertex adjustment values of the distortion mesh and applying the second set of vertex adjustment values of the distortion mesh are performed by at least one of: a (Pohl [0003] Pre-warping may typically be performed in a post-processing step using a pixel shader. However, the discrete resampling utilized by pixel shaders may lead to a loss in image quality. Further, some graphics architectures may not support the use of pixel shaders.) Ishihara in view of Pohl does not teach vertex shader. In a related field of endeavor, Selstad teaches: wherein applying the first set of vertex adjustment values of the distortion mesh and applying the second set of vertex adjustment values of the distortion mesh are performed by at least one of: a vertex shader or a fragment shader (Selstad [0049] Additional implementations of the technology disclosed include (i) each vertex of a distortion mesh observed by the cameras being independent, but traveling along a fixed set of optical elements, wherein the distortion mesh can be recomputed in a vertex shader of a graphical processing unit of the AR headset 100 ). Therefore, it would have been obvious before the effective filing date of the claimed invention to use a vertex shader as taught by Selstad . The rationale for doing so would have been that it is a simple substitution of one known element for another to obtain predictable results where Ishihara has a GPU performing warping, and Pohl has a pixel shader performing warping, where Selstad merely adds a vertex shader performing the processing, where the input of data to be warped and the output of a warped image is the same. Therefore it would have been obvious to combine Selstad with Ishihara in view of Pohl to obtain the invention. Regarding claim 13: The claim is a parallel version of claim 6. As such it is rejected under the same teachings. Regarding claim 20: The claim is a parallel version of claim 6. As such it is rejected under the same teachings. Conclusion For the prior art referenced and the prior art considered pertinent to Applicant’s disclosure but not relied upon, see PTO-892 “Notice of References Cited”. Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASON PRINGLE-PARKER whose telephone number is (571) 272-5690 and e-mail is jason.pringle-parker@uspto.gov. The examiner can normally be reached on 8:30am-5:00pm est Monday-Friday. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, King Poon can be reached on (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, seehttp://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /JASON A PRINGLE-PARKER/ Primary Examiner, Art Unit 2617
Read full office action

Prosecution Timeline

Jan 15, 2025
Application Filed
Oct 29, 2026
Non-Final Rejection mailed — §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12770332
THREE-DIMENSIONAL OBJECT REPRESENTATIONS BASED ON SURFACE POINT IDENTIFICATIONS
3y 4m to grant Granted Oct 20, 2026
Patent 12770537
PROJECTION CONTROL APPARATUS TO PROJECT OPTICAL ILLUSION RELATED TO CAUTION-RELATED FEATURE AND PROJECTION CONTROL METHOD
3y 0m to grant Granted Oct 20, 2026
Patent 12770302
SYSTEMS AND METHODS FOR DISPLAYING PATIENT DATA
2y 5m to grant Granted Oct 20, 2026
Patent 12755455
SYSTEMS AND METHODS FOR LAYERED IMAGE GENERATION
3y 1m to grant Granted Oct 06, 2026
Patent 12743184
USER INTERFACE MODES FOR THREE-DIMENSIONAL DISPLAY
3y 6m to grant Granted Sep 22, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
84%
Grant Probability
98%
With Interview (+13.7%)
2y 3m (~5m remaining)
Median Time to Grant
Low
PTA Risk
Based on 573 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month