DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is responsive to the correspondence filled on 07/15/2025.
Claims 1-20 are presented for examination.
IDS Considerations
The information disclosure statement (IDS) submitted on 10/16/2025 and 08/08/2025 is/are being considered by the examiner as the submission is in compliance with the provisions of 37 CFR 1.97.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission.
Claims 1 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 4 of US Pat. 11122251 B2.
Even though instant application does not claim “wherein each node position in the first UV map corresponds to a vertex of an environment-specific model; receiving, by the content playback device, first field of view (FOV) information corresponding to the first camera, wherein the first FOV information is indicative of a first field of view of an environment represented by the environment-specific model and associated with the first camera; receiving, by the content playback device, a first encoded image including image content captured by the first camera; decoding, by the content playback device, the first encoded image to generate a first decoded image; performing, by the content playback device, a first rendering operation on the first decoded image to generate a first image for display, wherein the first rendering operation is performed based on the first correction information, the first decoded image, the first UV map, the first FOV information, and the environment-specific model; receiving, by the content playback device”, however not claiming this does not provide instant application a patentable distinction. Because lack of limitation makes the claim broad obvious variation of US Pat. 11122251 B2.
Instant Application 19/270,251
US Pat. 11122251 B2
1. A non-transitory computer readable medium comprising computer readable code executable by one or more processors to:
receive, at a playback device,
first encoded stereoscopic image data captured by a first stereoscopic camera pair;
decode the first encoded image data to obtain a first decoded left eye image captured by a first camera and a first decoded right eye image captured by a second camera;
identify a first calibration [correction] profile for the first camera in accordance with the first decoded left eye image having been captured by the first camera; adjust the first decoded left eye image to obtain a first corrected left eye image using the first calibration profile;
identify a second calibration [correction] profile for the second camera in accordance with the first decoded right eye image having been captured by the second camera;
adjust the first decoded right eye image to obtain a first corrected right eye image using the second calibration profile; and
provide the corrected left eye image and the corrected right eye image for playback.
4. A content playback method comprising:
receiving, by a content playback device,
first correction information corresponding to a first camera, wherein the first and second cameras comprise a first stereoscopic camera pair,
wherein the first encoded image is a left eye image and the second encoded image is a right eye image.
wherein the first correction information is indicative of adjustments to node positions in a first UV map to compensate for image distortions introduced by a lens of the first camera, wherein the first encoded image is a left eye image
wherein each node position in the first UV map corresponds to a vertex of an environment-specific model; receiving, by the content playback device, first field of view (FOV) information corresponding to the first camera, wherein the first FOV information is indicative of a first field of view of an environment represented by the environment-specific model and associated with the first camera; receiving, by the content playback device, a first encoded image including image content captured by the first camera; decoding, by the content playback device, the first encoded image to generate a first decoded image; performing, by the content playback device, a first rendering operation on the first decoded image to generate a first image for display, wherein the first rendering operation is performed based on the first correction information, the first decoded image, the first UV map, the first FOV information, and the environment-specific model; receiving, by the content playback device,
second correction information corresponding to a second camera; receiving, by the content playback device, a second encoded image including image content captured by the second camera; and the second encoded image is a right eye image.
decoding, by the content playback device, the second encoded image to generate [adjust/correct] a second decoded image; and the second decoded image, the first UV map, the first FOV information and the environment-specific model; and the second encoded image is a right eye image, performing, by the content playback device, a second rendering operation on the second decoded image to generate a second image for display, wherein the second rendering operation is performed based on the second correction information,
wherein the first encoded image is a left eye image and the second encoded image is a right eye image.
9. Limitations of remaining claims of instant application are obvious over US Pat. 11122251 B2 in view of prior art discussed under Claim Rejections – 35 USC § 103 of this office action. Same motivation described under Claim Rejections – 35 USC § 103 of this office action is applicable for combining US Pat. 11122251 B2 and stated prior arts. Please note 35 U.S.C. 101 allows only one patent from one patent application or invention. Remaining dependent claim limitations points to same invention as per applicant disclosure. As such, all dependent claims of instant application are obvious variation of independent claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-8, 12-15 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Presler (U.S. Pub. No. 20100111489 A1), in view of Thomas (Immersive Panoramic Video - Thomas Pintaric, Ulrich Neumann & Albert Rizzo - Integrated Media Systems Center University of Southern California - Article . March 2002 DOI: 10.1145/354384.376408 . Source: CiteSeer - publication at: https://www.researchgate.net/pubication/2519331)
Regarding to claim 1, 8 and 15:
1. Presler teach a non-transitory computer readable medium comprising computer readable code executable by one or more processors to: (Presler [0042] The processor 36 of the processing sub-system executes reprogrammable software 38 that performs image processing for visualization, analysis, or storage. The processor 36 may be either dedicated hardware or general purpose central processing unit (CPU), graphics processing unit (GPU) or DSP or a combination thereof. Claim 14: 14. The digital camera system of claim 13, wherein the coded data is entirely stored on a removable storage media with a minimum sustained writing speed of 10 Megabytes per second.) receive, at a playback device, first encoded stereoscopic image data captured by (Presler [0058] The software 38 running on the processor 36 can code stereo streams as independent full-frame streams or can pre-combine the imagery into a single larger image or interleaved sequence for coding as a single stream. The metadata contained in the stream can be used to indicate the left and right image source and allow playback and editing of the stereo file as a single video source, yet displaying either source individually or as a mixed representation for stereographic display.) a first stereoscopic camera pair; (Presler [0115] FIG. 12 illustrates a flow chart 400 for the software 38 described above. The software 38 searches for cameras, including sensor units and camera modules, connected to the acquisition system via a data link, such as network, cameralink and HD-SDI with reverse serial communication and control channel ... The playback data may also be combined using image processing and visualization with live image preview images to perform functions such as keying of live action with previously recorded content. The live and playback processing can be done with either single source or multiple sources including stereo playback mixing with live stereo preview.)
decode the first encoded image data to obtain a first decoded (Presler [0057] The codec can combine data from image, audio and metadata and streaming metadata in headers of files and within groups of pictures (GOP) for use in decoding and editing. The coded data can be encapsulated into industry standard format file containers such Audio-video Interleaves (AVI), Quicktime (MOV). In playback, the codec, which may be on the same software platform or part of a post-production software program, can adaptively select to decode hierarchical resolution data, inherent in the wavelet transform, to enable real-time, multi-stream editing performance in software on standard PCs, without the need for specialized hardware. The compression or coding method may be software selectable for each recording and streaming function. The software 38 can also capture and record imagery as uncompressed data at various bit depths.) left eye image captured by a first camera and a first decoded right eye image captured by a second camera; (Presler FIG. 9 [0109] A first pair 186 of camera modules 54 (e.g., SI-2K MINI) are mounted for 3D stereo capture of a common scene thru an Edmund Scientific NT46-584 optical beam splitter 184. The two camera modules 54 are cabled together and are synchronized to capture at the same rate and to begin scanning with a common frame start time.) identify a first calibration profile for the first camera in accordance with the first decoded left eye image having been captured by the first camera; (Presler [0115] FIG. 12 The software uses then adjusts settings and calibration 404 the unique identifier to load previously generated and stored calibration data for use in the live acquisition video processing software pipeline 406. If the data does not exist, the software can initiate a calibration process to generate new calibration data. The software receives input settings 408 thru the camera hardware or from the software generated user interface, which may be touchscreen controls for adjusting functions such as camera settings, recorder settings, image processing modes, streaming and playback. The software continuously captures images and metadata 410, including audio, external device settings and operator settings. The image is then corrected, using the calibration parameters, including pixel-by-pixel black level correction and defect pixel replacement then packed 412 and placed into a RAM buffer 414 for additional processing)
adjust the first decoded right eye image to obtain a first corrected right eye image using the second calibration profile; (Presler [0009]Utilizing metadata encoded in the recorded stream or generated thru user input during playback the relative position, color transformation and format of the dual streams, representing the left and right eye content, can be adjusted to change the stereographic effect and depth perception on these displays. Presler [0025] As shown in FIGS. 1-12 Utilizing metadata encoded in the recorded stream or generated thru user input during playback the relative position, color transformation and format of the dual streams, representing the left and right eye content, can be adjusted to change the stereographic effect and depth perception on these displays. [0058] The software 38 running on the processor 36 can code stereo streams as independent full-frame streams or can pre-combine the imagery into a single larger image or interleaved sequence for coding as a single stream. The metadata contained in the stream can be used to indicate the left and right image source and allow playback and editing of the stereo file as a single video source, yet displaying either source individually or as a mixed representation for stereographic display.)
and provide the corrected left eye image and the corrected right eye image for playback. (Presler [0051] A particular configuration of a digital camera system 20 that uses multiple image sensor units 30 (via one or more modular camera units 28) input into a single processing sub-system 34 may be used for capturing multiple image simultaneously with the ability to synchronize the sources (i.e., sensor imaging units or camera modules), coordinate control and combine image processing, recording, display, storage and communication. This multiple camera configuration can be used for processing 3D stereographic and immersive scenes. The imagery (i.e., the RAW image data) and metadata (i.e., audio, positioning, timecode, etc.) from this multiple camera configuration can be recorded on a single removable storage medium or to independent storage devices in a synchronized fashion to enable simpler post-processing and display. The combined imagery can be outputted to specialized displays such as stereographic LCD monitors, 3D or spherical projection systems and VR goggles. [0060] The software 38 running on the processor 36 can generate processed display imagery from live or playback sources on single or dual outputs. The colorized raw or processed image data and metadata can be sent to a host (i.e., a separate computer which is not the mobile stereo recorder) for additional display, processing, recording or transmission. The software 38 can flip and mirror display imagery to enable a viewing system with two displays on a 3D beam splitter viewer. The display image data can be formatted and scaled for standard definition or high definition displays 44. For bayer image sources, the software 38 can be used to select the demosaic method, based on the available processing capability. The processed imagery may include generating stereographic displays including dual-streams image mixing, anaglyph, over-under, side-by-side, sequential switching and other modes, which may assist in perceiving the potential 3D stereo.)
Presler do not explicitly teach adjust the first decoded left eye image to obtain a first corrected left eye image using the first calibration profile; identify a second calibration profile for the second camera in accordance with the first decoded right eye image having been captured by the second camera;
However Thomas teach adjust the first decoded left eye image to obtain a first corrected left eye image using the first calibration profile; (Thomas page 6 para 1: Compression/ decompression would have to be added to allow remote player applications over broadband networks. In this live scenario the player application is responsible for correcting the camera’s lens distortion and stitching by adjusting the video texture’s UV coordinate mapping to reflect the camera calibration mapping described earlier (Fig.6). identify a second calibration profile for the second camera in accordance with the first decoded right eye image having been captured by the second camera; (Thomas page 3 para 2-4: Our approach is based on an array of multiple image sensors or cameras (Fig. 1). We make three observations about the advantages of multiple-sensor panoramic imaging systems over single sensor systems producing equivalent image quality. Firstly, at any point in time, the technology for a single high-resolution video image sensor is more expensive than multiple commodity sensors (e.g., the price of HDTV video cameras is currently more than an order of magnitude that of standard video cameras). Secondly, a single rectangular image sensor is inefficient for use in panoramic imaging since planar projections from curved mirrors or fish-eye lenses do not evenly or fully cover the pixel array (Fig. 2). Lastly, the use of multiple image sensors easily supports parallel approaches to managing the bandwidth and computation requirements in the system. The disadvantages of multiple sensors include the need for calibration and optics that produce a single shared viewpoint. The shared viewpoint is often only approximated in systems used with far-field scenes [6]. Another disadvantage that often occurs with multiple sensor designs is their limited vertical field of view (FOV). While our methods are not restricted to any particular camera array, we describe our system in the context of a commercially available five-camera array [5]. Other camera arrays have been built using seven cameras [6]. page 6 para 2: Fig 8 shows the player structure. We use the real time video texture capabilities found in current PC and gaming graphics systems. The cylindrical video projection produced by the camera is textured onto a matching 3D cylindrical model and the viewing is controlled by the tracker data [11] and a real-time adjustable scale parameter.)
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Presler, further incorporating Thomas in video/camera technology. One would be motivated to do so, to incorporate adjust the first decoded left eye image to obtain a first corrected left eye image using the first calibration profile; identify a second calibration profile for the second camera in accordance with the first decoded right eye image having been captured by the second camera. This functionality will improve efficiency with predictable results.
Regarding to claim 5, 12 and 19:
5. Presler teach the non-transitory computer readable medium of claim 1, wherein the first calibration profile and the second calibration profile are preloaded on the playback device prior to the playback device receiving the first stereoscopic image data. (Presler [0053] Upon identification, the software 38 can load image calibration data from a storage device 44 or can initiate a calibration process, which can extract data from the connected camera module(s) 28 including, without limitation, pixel-by-pixel black level, gains, shading, and defect pixels.)
Regarding to claim 6, 13 and 20:
6. Presler teach the non-transitory computer readable medium of claim 5, wherein the computer readable code to identify the first calibration profile comprises computer readable code to: detect an identifier for the first camera associated with the first decoded left eye image; retrieve the first calibration profile using the identifier for the first camera; and obtain the first correction information from the first calibration profile. (Presler [0115] FIG. 12 illustrates a flow chart 400 for the software 38 described above. The software 38 searches for cameras, including sensor units and camera modules, connected to the acquisition system via a data link, such as network, cameralink and HD-SDI with reverse serial communication and control channel. Once detected, the software identifies the cameras 402 by reading and determines make, model and unique serial number identifier, as well as any metadata stored in the camera or from devices which may be connected, such as sensors, optics, lens control and motion control systems. The software uses then adjusts settings and calibration 404 the unique identifier to load previously generated and stored calibration data for use in the live acquisition video processing software pipeline 406.)
Regarding to claim 7 and 14:
7. Presler teach the non-transitory computer readable medium of claim 1, wherein the first encoded stereoscopic image data is received as part of a live stream. (Presler [0114] FIG. 11 illustrates an alternate embodiment of the present invention where a digital camera system 300, similar to the digital camera systems 50, 80, 100, 120, 130, 140, 150, 180 and 200 described above, comprising a plurality of multiple camera modules, stereo rigs, mobile recorders and remote clients operating any of the subsystems via wired or wireless network, producing live 2D or 3D content for broadcast [live stream] or network streaming.)
Allowable subject matter
Regarding to claim 2-4, 9-11 and 16-18:
Claims 2-4, 9-11 and 16-18 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims because the limitations of these dependent claims are not obvious from the prior art search when all the limitations of independent and intervening claims are taken into account.
Regarding to claim 2, 9 and 16:
2. Presler teach the non-transitory computer readable medium of claim 1, wherein the first encoded stereoscopic image corresponds to a first point of view, and further comprising computer readable code to: receive, at the playback device, second encoded stereoscopic image data captured by a second stereoscopic camera pair, wherein the second encoded stereoscopic image data corresponds to a second point of view; (Presler FIG. 9 [0109] A first pair 186 of camera modules 54 (e.g., SI-2K MINI) are mounted for 3D stereo capture of a common scene thru an Edmund Scientific NT46-584 optical beam splitter 184. The two camera modules 54 are cabled together and are synchronized to capture at the same rate and to begin scanning with a common frame start time.)
Prior art do not teach decode the second encoded image data to obtain a second decoded left eye image captured by a third camera and a second decoded right eye image captured by a fourth camera; identify a third calibration profile for the third camera in accordance with the second decoded left eye image having been captured by the third camera; adjust the second decoded left eye image to obtain a second corrected left eye image using the third calibration profile; identify a fourth calibration profile for the fourth camera in accordance with the second decoded right eye image having been captured by the fourth camera; adjust the second decoded right eye image to obtain a second corrected right eye image using the fourth calibration profile; and provide the second corrected left eye image and the second corrected right eye image for playback.
Closely related prior art
Examiner notes teaching of U.S. Pub. No. 8963916 B2 is/are pertinent to the independent claim(s), however is not used because dependent claims are better covered by cited reference.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NASIM N NIRJHAR whose telephone number is (571) 272-3792. The examiner can normally be reached on Monday - Friday, 8 am to 5 pm ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William F Kraig can be reached on (571) 272-8660. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NASIM N NIRJHAR/Primary Examiner, Art Unit 2896
Prosecution Insights